Knoppix Debian Integration

> Is there a process for making these CDs from nothing more than debs
> downloaded from debian.org?

If it is Morphix then this is not possible or intended. However Morphix is n fact a good base to get this done.

> Obviously it's non-trivial, but it should be possible to have debootstrap
> (or a similar tool) construct bootable Knoppix-esque CDs without any
> user-interaction IMO.

I hope that people like the Morphix developer(s) will grab the idea to do so. In fact it is more than trivial but in the end it would make building customized Knoppix CDs trivial which is the goal of Morphix (if I understand the web page right).

Andreas Tille

> Is there a process for making these CDs from nothing more than debs
> downloaded from debian.org?

I know at least of one such process and its the one used at metadistros (metadistros.hispalinux.es). The tool in this case is called 'calzador'. IIRC it basicly depends on having a chroot environment set with the OS image you want to burn and it adds that together with the bootup stuff.

Javi

I am the author of Debix and some might already know me from the Linuxtag or the Oldenburg D-I Debcamp.

First, before you ask, Debix developement is currently on hold in favour of working on D-I but D-I is getting to a stage where it can utilize Debix so it will start back up soon.

Debix is ment as a tool to make any linux system into a live CD with no change to the linux system. With the help of the device mapper (kernel part of lvm2) and an initrd a (compressed) loopback file on the cdrom (or other medium) is made seemingly writeable. Any changes to the loopback file will be stored in ram.

Using Debix you can take a completly normal Debian system created with debootstrap and make it a live CD. Added features are that you can move the data from the cdrom to any other blockdevice in the background while the user is working to make a harddisk install for example. The user would just provide the partition and at some time later the cdrom would eject and the install is done and already running, allways had been. Given a cdrom and a burner you can update the running system (apt-get update; apt-get dist-upgrade) and reburn it.

Why do I tell you this?

Debian will soon have Debix to make transparent live CDs. All you have to worry is getting the autodetection and autoconfiguration into policy conform debs that makes your live CD so great. Once you can install your system on a harddisk or chroot or update Debian to such a system your ready to make a live CD with Debix.

Goswin von Brederlow

> > Debian will soon have Debix to make transparent live CDs. All you have
> > to worry is getting the autodetection and autoconfiguration into
> > policy conform debs that makes your live CD so great. Once you can
>
> I'm not keen on the name Knoppix.  But we have to move all the nice
> auto-detection stuff of Knoppix into this build system.  Moreover there
> might be some help in preconfiguration of desktop etc, which is more
> or less done in Knoppix (or Gnoppix for Gnome).  People might say that
> some stuff in Knoppix might violate Debian policy.  But I see no direct
> relevance of Debian policy for a derived LiveCD as long as the
> preparation is done in a clean chroot ... and this violation is
> necessary / makes sense.

Acutally no. Debix will not get any of that.

Its your job as the user of debix to provide a already "knopified" chroot or to provide debs that debix will install via debootstrap to create a filesystem for the cd.

Producing debs for this would be better because then you can easily update and modify the system. Making the debs policy cleans allows them to enter debian. The debs should be such that they can be used on a normal debian harddisk installation. Debix takes care of the live cd stuff transparently. This also mean any debian user can just upgrade his debian to a knoppix.

If can't work with the policy but still want something in debian you would have to make a package that can create a knoppix chroot, i.e. package the scripts that transform a "normal" debian to knoppix. But I strongly suggest building policy conform debs for everything even if it means some things become slightly more complex (or bend the policy).

I suggest putting all the knoppix debs into task knoppix. That way users can install debian and when task-sel pops up they select task knoppix, wait some and then they have they fully featured knoppix on the harddisk. Not as smooth as the live CD but still much easier than configuring everything themself.

Goswin

Andreas Tille <til…@rki.de> writes:

> > If knoppix is all handled by debs you can koppify your installed
> > system by "apt-get install <knoppix debs>". Thats the end goal of
> > getting knoppix into Debian in my opinion.
>
> So we have very different opinions about Knoppix.  I understand
> Knoppix as plain Debian prepared as live CD while it has some nice
> preconfiguration but this is more or less debian-desktop work and
> has absolutely nothing to do with Knoppix.

Debix (through the device mapper) completly removes the live CD aspect of the installation. The CD part becomes completly transparent. The Knoppix developers are quite intrested in this since all the other methods (onionfs, lucientfs, symlink farm) all, well, suck. When 2.6 is released and becomes stable device mapper support is in the standard kernel so no extra patches will be neccessary, which is another argument for it.

> > For the same reason that users want the hd-install function on the
> > knoppix.
> I see this as workaround to get a nice Debian installer.  This will
> be mostly fixed by d-i (hopefully).

D-I doesn't realy go that way. Its just a nicer, modular design of boot-floppies. You can add nice modules to it to do this kind of work but it a different approach from a live cd. D-I does an install from scratch with a minimal system (the initrd on the floppy or boot image) using the udebs and then another install of base onto the target medium.

Goswin

> > the installation. The CD part becomes completly transparent. The Knoppix
> > developers are quite intrested in this since all the other methods
> > (onionfs, lucientfs, symlink farm) all, well, suck. When 2.6 is released
> > and becomes stable device mapper support is in the standard kernel so no
> > extra patches will be neccessary, which is another argument for it.

> So am I to understand that you plan to use device-mapper snapshots in order
> to provide copy-on-write at the block device level in order to implement
> this?  If so, this is interesting.  I assume you will use a RAM disk for
> writable storage?  What filesystem will you use?

Yes, thats exactly how it works. One device for the loopback file, one for the ramdisk/tmpfs storage and a copy-on-write snapshot combining those two into the real root device.

Recent kernels allow using loopback files on tmpfs. Older kernels need to use one or more ramdisks.

The problem with ramdisks is that the default size is 4 MB and specifying the ramdisk size as boot parameter is not allways possible. Compiling it in is also bad. With tmpfs the size can be set at runtime and if someone adds swap that will be used for the tmpfs as well when needed.

As filesystem I only used ext2. But device mapper provides a normal block device like any other. Only think to note is that using a journaled filesystem like ext3, xfs or reiserfs might not be the best idea since the journal will fill up 32 MB (or whatever the fs has) of ram with no benefit.

On the other hand if harddisk install is to be suported using xfs would have the big advanted that one can move the device snapshot onto the harddisk and grow the filesystem to fit. The moving is done with a kernel thread beneeth the block device and at some point the data will have moved from the cdrom and ramdisk to harddisk and the cdrom/ramdisk can be ejected/freed. You never have to reboot or otherwise stop your work. At some point it will just tell you its done.

Goswin

> > If knoppix is all handled by debs you can koppify your installed
> > system by "apt-get install <knoppix debs>". Thats the end goal of
> > getting knoppix into Debian in my opinion.
> So we have very different opinions about Knoppix.  I understand
> Knoppix as plain Debian prepared as live CD while it has some nice
> preconfiguration but this is more or less debian-desktop work and
> has absolutely nothing to do with Knoppix.

Ummm… You are wrong here. Knoppix (or Knoppix-derived versions) provide three things:

The (3) part is not something that debian-desktop will do since it boils down to modifying, at leisure, the system's configuration (/etc directly, since there is not a single point of configuration, debconf is not an option here). This preconfiguration is the one that allows customisation for the system to end-users and it basicly boils down to doing it on your own system (touching files, tweaking stuff, adding tools, etc.) and then using it as an image for the live-CD (1) and as an 'image' for user installations (4).

It's also something that cannot be 'packaged' easily (there are as many possible setups as users out there), and usually this packaging violates policy (since the package would modify other packages configuration files).

The nice part of Knoppix is not that it auto-detects hardware since this can be done already in Debian (kudzu and the like are already available) but that somebody has taken the time to customize the environment (the desktop or whatever) to suit the needs of an specific group. He has selected the tools that will be available and has customised them to work as expect.

For example, take a spanish user group for example with ~0 computer administration knowledge that want an Internet environment. You can provide a system that is a replica of yours which is already configured enabling spanish i18n everywhere (task that involves modifying /etc/ configuration files), with Mozilla already configure to work in spanish (see #192293), bookmarks for common spanish sites, with the desktop already configured simulating Windows behaviour, with xchat already configured with spanish IRC channels, etc.

Hope that clears up the idea

Javi

> d-i is perfect for a custom installation, it's never going to be a system
> in which everything is already predefined and the user just says 'install
> this'.

er, check out skolelinux, which uses d-i and debconf pre-seeding to do more or less exactly this (not everything can be configured through debconf at the moment)

cobaco

Everything that follows is inspired by or lifted directly from conversations with Skolelinux folks like pere.

On Wed, Nov 26, 2003 at 02:38:38PM +0100, Javier wrote:

> Knoppix (or Knoppix-derived versions) provide three things:

<snip>

> 3. preconfiguration of all the software and customisation for a specific
> need.

<snip>

> The (3) part is not something that debian-desktop will do since it
> boils down to modifying, at leisure, the system's configuration
> (/etc directly, since there is not a single point of configuration,
> debconf is not an option here).

I'm not sure you can speak so categorically about this. Skolelinux has, in some ways AUIU, gone this route already.

What we *can* do is find the ways that we, as a custom distribution, want to change the configuration files of other packages and then submit wishlist bugs with patches adding low-priority debconf questions with defaults set to the current behavior. Now the package need never even *ask* those questions!

Now, when someone does a Custom Distro installation, they'll have a CD that is vanilla Debian plus some presets to fill the Debconf database with the values you want for your new debconf questions.

> It's also something that cannot be 'packaged' easily (there are as many
> possible setups as users out there), and usually this packaging violates
> policy (since the package would modify other packages configuration
> files).

This is the beauty of the method above. I'm sure there are limitations on this model but if we follow this sort of plan, it's not difficult to imagine a time in the very near future when every Custom distribution is pure Debian plus a single policy non-compliant package and the goal of every custom distro to reduce that the size of the package to zero. :)

Benjamin Mako Hill

> > We don't want to use, as a distribution, a single point of
> > configuration like debconf. I might be wrong or things might have
> > changed.
> I thought consensus was pretty good that we *do* want to have a single,
> standardized interface for package configuration wherever possible.

Erm, yes, but isn't that "/etc" ?

Certainly we're not planning on replacing /etc/exim/exim.conf with various debconf scripts. I can't imagine that we want to drop things like "webmin" either, which offers an alternative interface for package configuration.

Debconf is, at most, a single, standardized interface for interaction to ensure that packages are minimally configured for operation at install.

Personally, I'm leaning towards thinking that if you have some special configuration for a package you want people to be able to use you should do something like:

In particular, that doesn't require the package to change every time some random derived distribution wants to configure it in some particular way.

Anthony Towns <a…@humbug.org.au>

> The easiest way for a debian-live-cd is, run cdbackup.
> There a lot of interesting projects based on debian, ex. gibraltar,
> debian 4 Xbox, Lindows, all of them are debianbased. Impossible to
> include all them into debian. I think its better to help those projects,
> bring them closer to debian. Find out what is different compared to a
> deb. Xfree is the same piece of software. I'm quit sure you can find
> every software from knoppix inside debian.

> Technically you boot a ( i can't understand why ) modified debian-base

With debix you start the debix kernel (kernel-image with kernel-patch-device-mapper or a 2.6.x kernel when it comes out) with an initrd. The initrd detects some hardware to find the cdrom, mount it and sets up a snapshot over the loopback file on cdrom. That in turn is then mounted, /etc/fstab adapted to the current mountpoints, pivot_rooted and chrooted into and /sbin/init started like any normal boot.

I have a cd image with a 100% debian woody base system created with the boot-floppies from woody as the loopback file. Works perfectly. Thats the goal of debix, a 0 change live CD creation.

> system and start automatic hardware-configuration then your Desktop/
> Firewall/ File-server/ Cluster. I do not understand differentiated to a
> debian. A cool/colorful Desktop ?

The important part is autodetection and autoconfiguration of hardware. Configuring say X in debian is a pain since 99.9% of all people can use the autogenerated config from say knoppix.

Basically that autoconfiguration needs to be merged into the debs but the maintainer is not realy open for this. And writing a second deb to configure X would be against policy.

Whats left? Fork the xfree deb? No way.

For the xfree-autoconf package (or whatever it will be called) I probably have to dpkg-divert the X server and replace it with a wraper that makes it use different config files, those generated by xfree-autoconf. Thats quite ugly.

Goswin

> Technically you boot a ( i can't understand why ) modified debian-base
> system and start automatic hardware-configuration then your Desktop/
> Firewall/ File-server/ Cluster. I do not understand differentiated to a
> debian. A cool/colorful Desktop ?

I have no problem with Debian derived distributions and I'm sure people who are doing this have thought about it. But currently Debian is missing something: We have no brain dead method to create a user customized CD with really good auto detection. IMHO, Knoppix and its derivatives might fill this gap. That's why my idea was to reintegrate Knoppix but I would have no problem if somebody would like to do it from scratch.

The popularity of Knoppix speaks for what users really like. The fact that there are so many Knoppix derivatives speaks for the need of customizing a live CD which is currently not as easy as users might deserve it.

Andreas Tille

> > ..  But currently Debian is missing
> > something: We have no brain dead method to create a user customized CD
> > with really good auto detection. ..
> [snip]

> What things are autodetected by Knoppix and its ilk?

Two major things I can name:

kernel modules (apt-get install discover(2) or hotplug takes care of that)

XFree configuration

> What things are not autodetected?

> What things are autodetected in "compatibility mode", i.e. will likely
> require configuration tweaks for maximum performance?

XFree might fail to work perfectly or might need a bootparameter to use a framebufer driver instead. But I've heard far more people on irc saying knoppix works but debian they can't get working than the other way around.

> What things are so easy to autodetect that they are "just right every
> time"?

> What's the ratio of things in each category?

> Why don't you just move autodetection into the main kernel or driver trees?

Keep stuff out of the kernel. Its too big already. We don't need or want autodetection of the monitor frequencies or the dsl contracktor in the kernel.

> I don't see why it needs to be this complicated...

Three things: 1. thick headed maintainers, 2. policy, 3. time

Goswin

>It would be great if Debian could incorporate its hardware detection ...
>but if so, then it should be the Debian way. This way standard
>installation could profit from the autodetection too. All the other
>startup tricks have to be looked at and incorporated into their
>corresponding Debian package if they are better/easier to use.
>(hotplug, pcmcia, automounts, etc. etc.)

I fully agree. *Everybody* should look at http://developer.linuxtag.net/knoppix/sources/, check if there are any packages related to his own and try to merge whatever is in knoppix.

Marco d'Itri

>I'm not keen on the name Knoppix.  But we have to move all the nice
>auto-detection stuff of Knoppix into this build system.  Moreover there

A big part of the autodetection is in the hotplug package, or at least it would be if the upstream had not butchered it. My patched hotplug with a 2.6 kernel is able to recognize almost all hardware for which a kernel module has been installed, and automatically load the module needed to use it.

Marco d'Itri

> > There's still much to do in order to develop said script but some projects
> > were trying to automate this process. It still remains to see if these
> > scripts are going to be reusable and could be included in Debian.
> Why?
> Any piece of free software might be able to include into Debian.

Thats true, but integrating Knoppix into Debian basically boils down to rewriting it, because it violates Debian policy in too many cases. Actually I think it would be worth the effort to do that.

Guenter

My objective is to install a standard Debian system on the target HD and have made useful progress towards this. I have found that this needs to be done in two stages.

The problem is that the packages on the CD have to be compatable with the specialised Knoppix packages. KDE is giving me a headache at the moment.

IMHO, the way forward is to create an independant Knoppix sub-system which can be removed after the installation rather than the Knoppix utilities relying the packages which are to be installed. However, this is going to use CD space.

My just completed iso image is 723MB, back to dselect.

Phil.

p.s. I am using "standard Debian system" to mean an installation that can be manipulated normally by apt etc.

We definitely nead a feature to create customized live CDs *easily*. I'm absolutely convinced that the best way to do it *right* is to include this into official Debian distribution. If such kind of live CD shows other nifty features I would be happy.

> > (May be if we will be able to install from a live CD later but the current
> >  hack can not be used for this purpose.  The goal has to be a clean Debian
> >  install and no Knoppix-like hack).

> since knoppix 3.3 there is one more hdd installer along with knx-hdinstall, it
> is called knoppix-installer and has mode=knoppix and mode=debian. I have
> never tested it, but I guess it installs more policy compliant Debian system
> on hdd.

Well, I guess the difference is like between a "dirty hack" and a "not so dirty hack".

Again and again: I do not care about how cleanly a live CD can be installed on a random box. What I need is an easy way to *build* a live CD. Just assume that my only purpose is to run it on diskless systems if it is so hard to understand that we need the process of building the CD first before we think about how it can be installed anywhere (which would be nice but absolutely not necessary because we have a way to install Debian).

Andreas Tille

> Again and again: I do not care about how cleanly a live CD can be
> installed on a random box.  What I need is an easy way to *build* a
> live CD.

For that debix is ideal. You can start of with a base live cd, boot, configure, install, customize. The you burn a new CD with an image of the snapshot (i.e. all your changes) or you copy an image of the snapshot over net.

This reburning of the running debix system is what I'm working on right now. I find that much easier than changing a chroot, burn it, try it, reboot and start again for the next typo.

Goswin

> Could you please define precisely "flavours" and "derivative distros"?

Damn, I thought I'd already done that.

Since I evidently didn't, I'm going to spell things out in as much boring detail as I can. If I don't end up insulting your intelligence, my apologies. :)

> I see no problems in documenting that the name "Custom Debian" includes
> "Flavours" and "Derivative Distros", and then define what they are.

Okay, IMO there are two techniques worth distinguishing that both aim to achieve the same goal. That goal is to put a CD (or DVD, or mirror) in the hands of a user, who can use that CD to get a running, dpkg-based system that does a particular thing s/he wants, and nothing else. If s/he wants to setup a multimedia box that records TV programs and acts as an mp3/ogg jukebox, that's the only sort of software s/he sees — no web proxies, no intrusion detection tools, no compilers, no documentation on setting up RAID arrays and hot-failover. If s/he wants to setup a fileserver for a small business, s/he might see Samba and Appletalk and NFS, but there won't be any games or scientific analysis tools available.

The issue isn't so much one of removing those tools entirely — ideally, they'll just be an apt-get away anyway — but of putting the things that are actually wanted on the first CD (or at least the first few CDs), and making the installation and configuration process as quick and easy as possible.

I think "Customized Debian" is a good name for that sort of thing — it's Debian, but it's customised for particular usage scenarios. If the usage scenario is common enough, then that's a real win: one person can do the customisation, and hundreds or thousands can reap the benefits.

> On the other end, I feel that if you see Flavours and Derivative Distros
> as subsets of Custom Debians, then we might have different concepts in
> mind.

Now, there are different possible approaches to this. The most flexible is to create a "Debian derivative" — that is, to take Debian, pull out the bits you like from it, upgrade some things, downgrade others, recompile some of the stuff that doesn't work quite right, improve a few things, and add some completely new stuff. That's great, and it's been demonstrated to work quite well.

The problems are straightforward: if you're writing your own stuff, you have to manage your own security updates. If you're forked from Debian, then Debian might make some changes that break compatability with your stuff, and you might have to think a bit to integrate the changes. You'll need to find someone to host your images. You can't leverage most of the Debian infrastructure (BTS, autobuilders in particular, probably).

But there are plenty of benefits too: you don't have to worry about non-i386 if you don't want. You can set your own policies and not have to answer to anyone, or convince anyone they're good. You can set your own schedule. If you've got software that Debian can't distribute (it costs money per copy or it's internal-use-only, eg), you have to go this way, to some extent.

So that's what I call a Debian derivative. It's obviously a customised Debian, but it's customised by taking Debian and adding stuff to it.

While that's by far the most effective way of customising Debian, it's not the only conceivable way. The other conceivable way to customise Debian is just to look at all the packages, and rm the ones you don't care about. If that gives you want you want, you overcome a bunch of the more annoying shortcomings above: you don't have to do your own security updates, you don't have to arrange your own hosting, new upstream releases will be packaged for you without you having to lift a finger, it'll normally work on all supported architectures without any effort, and you can file bugs in the BTS without anyone caring that you didn't install from an *official* Debian CD.

That's what I'm calling a "flavour" of Debian. A different analogy which makes a little more pedagogical sense is to consider it a "shade" of Debian — making the analogy with colours and prisms instead of taste. Debian, the universal operating system, beams white light at you all day long, and you put a prism or a filter in its way to get just the "shade" of Debian that you want. We do this already by choosing which packages we want on individual systems and setting them up, which is fine; but what we really want is to be able get a pre-fab filter from the store, and just plonk it in, so we don't have to bother ourselves all the time.

We can do that to some extent at the moment — with sections and tasks and fai classes eg. Which is okay, but it's *far* beneath the level of coolness provided by Knoppix. And as well, if we get flavours to work almost as well as Knoppix (creating a livecd that autodetects hardware and sets you up in a Linux environment with KDE and Gnome and whatever else, by telling it nothing more than which bits of Debian you want on that livecd), then that makes maintaining Knoppix a lot easier, since half the work is already done.

> With my ideas of Flavours and Derivative Distros, Flavours are Custom
> Debians, but Derivative Distros are things like Knoppix which are
> derived from Debian but are not policy compliant, and so they can't be
> called Custom Debians.

I'm not seeing any difference between "flavour" and "custom Debian" in the above then. I think it's a waste to have words that mean /exactly/ the same thing.

So, using my definitions, the following conclusions are (IMO) true:

Anthony Towns

On Tue, 2003-12-02 at 02:46, Anthony Towns wrote:

> So, using my definitions, the following conclusions are (IMO) true:

Awesome summary.

That's great - it covers all bases, and it makes sense to me (as someone who hasn't used the other terms (internal proj, subproj, metadistro, etc) in the past. I think it would be useful for this to be added to the Custom Debian Distro wiki (here I think: http://wiki.debian.net/index.cgi?CustomDebian).

Thanks heaps Zen

 Package: debix
 Architecture: any
Description: Create a live filesystem from an existing system or scratch
 Debix can create a live filesystem from any existing linux system or
 create a fresh system with debootstrap. By using the device mapper
 from LVM 2 the live filesystem only needs a new initial ramdisk and
 otherwise uses the existing system unchanged as loopback image.
 When creating a live filesystem from scratch knoppix like
 autodetection features and the official debian boot-floppies and/or
 debian-installer can be included to make a more confortable
 installation.

There is debix-imager package also … and this is completely on the topic which reads 'Integrate Knoppix in Debian'.

Feel free to check it out:

cvs -d:pserver:anonym...@cvs.alioth.debian.org:/cvsroot/debix login
cvs -z3 -d:pserver:anonym...@cvs.alioth.debian.org:/cvsroot/debix co .

The official debian boot-floppies and/or debian-installer could be used, there's nothing wrong to explore b-f & d-i in this way. The so called 'new generation' here is 'Create a live filesystem from an existing system or from scratch' on-user-demand.

George Danchev

> > Is there a process for making these CDs from nothing more than debs
> > downloaded from debian.org?
>
> I know at least of one such process and its the one used at metadistros
> (metadistros.hispalinux.es). The tool in this case is called 'calzador'.

Is there any code that goes with that? I saw the wiki, but not knowing any Spanish, didn't really get much from it.

> IIRC it basicly depends on having a chroot environment set with the OS
> image you want to burn and it adds that together with  the bootup stuff.
> > Obviously it's non-trivial, but it should be possible to have debootstrap
> > (or a similar tool) construct bootable Knoppix-esque CDs without any
> > user-interaction IMO.

I'm envisaging something like:

mkdir sarge-chroot
debootstrap --include knoppix,calzador sarge sarge-chroot
        # setup a chroot environment with knoppix and calzador tools
chroot sarge-chroot /usr/sbin/knoppix install-pkgs
        # install and configure the standard "knoppix" patches
calzador setup-for-readonly sarge-chroot
        # reconfigure /etc so chroot can be booted on a readonly
        # root
mkisofs -b /usr/lib/calzador/boot_image -o myknoppix.iso sarge-chroot
        # create an bootable isofs image
cdrecord ...
        # then burn it to a CD

Being able to automatically install a useful system with lots of gnome and kde and whatever other stuff without having to spend hours in dselect or aptitude or answering questions or doing other configuration stuff has obvious benefits, as does being able to easily convert an existing system into something you can burn to a CD.

Also obvious, though, is that both those steps are hard: working out a good set of packages to install, and configuring them usefully is non-trivial; as is making Debian work with a read-only root.

(While the hypothetical /usr/sbin/knoppix and /usr/sbin/tasksel both solve similar problems they're not quite the same — /usr/sbin/knoppix needs to select _and_ configure all the packages you want, tasksel just selects them. debconf might not be enough either — /usr/sbin/knoppix might need to actually modify various files in /etc, too)

Anthony Towns

> > > Is there a process for making these CDs from nothing more than debs
> > > downloaded from debian.org?
> > I know at least of one such process and its the one used at metadistros
> > (metadistros.hispalinux.es). The tool in this case is called 'calzador'.

Looking at the question again I see that I did not answer the question properly. The process used by Metadistros is not based exclusively on Debian downloaded debs, but uses some custom tools/software. The 'calzador' is really just part of the process and is really a set of preconfiguration data/scripts for use in a LiveCD.

> Is there any code that goes with that? I saw the wiki, but not knowing any
> Spanish, didn't really get much from it.

Yes, maybe not too ready for mainstream: ftp://ftp.softwarelibre.ulpgc.es/METADISTROS/calzador/calzador.tar.bz2

the (modified) kernel modules for Live-CD support are available at ftp://ftp.softwarelibre.ulpgc.es/METADISTROS/calzador/modulos.tar.bz2

The procedure is not yet completely automated (I got it wrong, sorry). It is described (in Spanish) at http://metadistros.hispalinux.es/tiki-index.php?page=HowTo

> I'm envisaging something like:

>    mkdir sarge-chroot
>    debootstrap --include knoppix,calzador sarge sarge-chroot
>            # setup a chroot environment with knoppix and calzador tools
>    chroot sarge-chroot /usr/sbin/knoppix install-pkgs
>            # install and configure the standard "knoppix" patches

This step in meta-distros is based on a meta-package that downloads all the needed packages and dependancies.

(…)

> Being able to automatically install a useful system with lots of  gnome
> and kde and whatever other stuff without having to spend hours in dselect
> or aptitude or answering questions or doing other configuration stuff
> has obvious benefits, as does being able to easily convert an existing
> system into something you can burn to a CD.

That's really what has made interest in Knoppix-like distributions spark.

> Also obvious, though, is that both those steps are hard: working out
> a good set of packages to install, and configuring them usefully is
> non-trivial; as is making Debian work with a read-only root.

That's why people usually 'remaster' Knoppix instead of starting from scratch. That is, they take the live-CD, dump it to disk, chroot into it, remove/add stuff and generate a new liveCd.

> (While the hypothetical /usr/sbin/knoppix and /usr/sbin/tasksel both
> solve similar problems they're not quite the same -- /usr/sbin/knoppix
> needs to select _and_ configure all the packages you want, tasksel just
> selects them. debconf might not be enough either -- /usr/sbin/knoppix
> might need to actually modify various files in /etc, too)

Taken into account the configuration stuff in Live-CDs is usually done manually and is also very specific to the kind of live-CD you want to create (as well as who will use it, what applications you want to include, etc..). I don't believe that should be included in a /usr/sbin/knoppix. I might be wrong however.

Regards

Javi

> Taken into account the configuration stuff in Live-CDs is usually done
> manually and is also very specific to the kind of live-CD you want to
> create (as well as who will use it, what applications you want to include,
> etc..). I don't believe that should be included in a /usr/sbin/knoppix.
> I might be wrong however.

Maybe I should've called it /usr/sbin/klaus_knopper ;)

Anyway, the stuff that hypothetical binary does is:

The first is trivial to automate once you've selected the packages, although it can suck a bit to maintain as some of the packages get dropped from unstable.

The second is harder to automate, but it's something we need to do anyway: it's the same thing we want to be able to dump images out to a hundred computers in a university lab, or a thousand computers on secretaries desks at your favourite multinational or government agency, or ten-thousand computers in a roll-out for a new search engine.

Getting that working for a single case (live CDs), is a good way to get it working well for the general case, and even if not, tends to make the single case a bunch easier.

Anthony Towns

> I'm envisaging something like:

>    mkdir sarge-chroot
>    debootstrap --include knoppix,calzador sarge sarge-chroot
>            # setup a chroot environment with knoppix and calzador tools
>    chroot sarge-chroot /usr/sbin/knoppix install-pkgs
>            # install and configure the standard "knoppix" patches

This remains to be done.

>    calzador setup-for-readonly sarge-chroot
>            # reconfigure /etc so chroot can be booted on a readonly
>            # root
>    mkisofs -b /usr/lib/calzador/boot_image -o myknoppix.iso sarge-chroot
>            # create an bootable isofs image

This becomes: debix —from-dir sarge-chroot myknoppix.iso

Which internally does (simplified):

mkdir myknoppix cp kernel initrd bootloader myknoppix genext2fs -x myknoppix/debix.img -d sarge-chroot mkisofs -o myknoppix.iso myknoppix

The beauty of using device mapper (which operates on the block device level) is that its transparent to the system.

> Also obvious, though, is that both those steps are hard: working out
> a good set of packages to install, and configuring them usefully is
> non-trivial; as is making Debian work with a read-only root.

Debian only has a very few things left that need to be changed for a read-only root. All software should be changed to allow for linking files or dirs into a ramdisk or to a kernel file (/etc/mtab -> /proc/mounts). If you know of any that can't cope a link to a writeable place let me know and file a bug.

Also with debix you don't have a read-only root unless you want to.

> (While the hypothetical /usr/sbin/knoppix and /usr/sbin/tasksel both
> solve similar problems they're not quite the same -- /usr/sbin/knoppix
> needs to select _and_ configure all the packages you want, tasksel just
> selects them. debconf might not be enough either -- /usr/sbin/knoppix
> might need to actually modify various files in /etc, too)

Idealy debconf should be enough or a wraper package should be available. Using only policy conform debs is my goal for a knopix integration. Too many people have come into irc and complained that updating knoppix to debian/sid broke everything. If knoppix actually were part of debian that would be very bad.

Goswin

Re: debix etc. These sound awfully like mkinitrd-cd, which already does all the work of cdrom detection, pivot_root etc. We've already got so many tools and groups working to accomplish the same thing, it would make more sense to agree on the goals and work together. I recommend using FAI as the installer and use its class system, until d-i is ready for similar hooks. Then it's just a matter of each group going off and defining their meta-distribution classes, we all use the same live CD creation script, same "install" method etc.

Niall Young

> Also with debix you don't have a read-only root unless you want to.

Info: I have a page that summarizes read-only root issues:

http://panopticon.csustan.edu/thood/readonly-root.html

 Re: my "allow mtab to be at the end of a symlink" patch ...
The patch was acknowledged by the util-linux upstream maintainer.
No promise was made that the patch would be applied, however.

Thomas Hood <jdth…@yahoo.co.uk>

> Info: I have a page that summarizes read-only root issues:
>     http://panopticon.csustan.edu/thood/readonly-root.html[]

This isn't a general solution, but it is enough to get livecd's working (where you already know you're mounting /var as a ramdisk).

I do wonder if we wouldn't be better off just saying "/var will be mounted by the time /etc/rcS.d/S40networking runs", even if that means systems that NFS mount /var have to have special networking scripts they use to do that. That makes the common case (where /var is a local disk or a ramdisk) easy, but the uncommon case is still possible.

Anthony Towns

> This isn't a general solution, but it is enough to get livecd's working
> (where you already know you're mounting /var as a ramdisk).

> I do wonder if we wouldn't be better off just saying "/var will be mounted
> by the time /etc/rcS.d/S40networking runs", even if that means systems
> that NFS mount /var have to have special networking scripts they use
> to do that. That makes the common case (where /var is a local disk or
> a ramdisk) easy, but the uncommon case is still possible.

As I said, debix can take care of all that on a much lower and earlier level so that should not concern.

Goswin