First create a directory tree containing compressed files:

mkzftree input_dir compressed_dir

mkzftree will not overwrite an existing directory; you may want to "rm -rf" the directory tree if you are doing this from a script:

Second, invoke the patched mkisofs with the -z option:

mkisofs -z -R [other options] -o compressed.iso compressed_dir

-z, -transparent-compression
                            Enable transparent compression of files

-R, -rock                   Generate Rock Ridge directory information

Note: