[Gpg4win-devel] claws extensions

[Marcus Brinkmann]

At Tue, 14 Oct 2008 18:03:16 +0200,
Bernhard Reiter wrote:
> [1  <multipart/signed (7bit)>]
> [1.1  <text/plain; utf-8 (quoted-printable)>]
> On Dienstag, 14. Oktober 2008, Marcus Brinkmann wrote:
> > > given that claws grows in complexity, at one point in the future we
> > > should consider a seperate claws installer that needs Gpg4win-X to do
> > > crypto.
> 
> > In addition to what Werner said, what you propose only makes sense if
> > a proper packaging system is used that can automatically resolve such
> > dependencies, and then all of gpg4win should be split into individual
> > packages, much like Debian, but with a simpler GUI.
> 
> I am not worried about size, I am worried about complexity.

Well, in this case that's a false alarm.  The packaging of claws is
the same as that of other source packages.

<technical stuff>
In fact, there is nothing complex in gpg4win going on.  Gpg4win has
two layers: One layer is the auto-builder, which pulls the up-to-date
source and binary packages and builds/installs them in a temporary
directory.  The second layer is the NSIS installer.  Both are
completely separate.

* The auto-builder is very simplistic.  We only support four or so
  package formats, and for each we have a fixed sequence of commands
  we execute, of the kind "configure; make; make install; stow".

  There is some trickery in providing a sane cross-compilation
  environment, most notably with pkg-config, which needs some hacks.
  But these issues are entirely localized to this problem.

  The auto-builder comes in the files
  configure, m4/gpg4win.m4, packages/*, patches/*, src/gpg4win.m4.in
  src/Makefile.am, src/pkg-config

  If you want to know how simple this autobuilder core really is,
  check out "speedo" at cvs.gnupg.org:/cvs/mb/speedo, which is what I
  use to build the gpg packages from SVN.  It uses the main part of
  the auto-builder, gpg4win.m4.in and Makefile.am (as we don't use
  pkg-config):

  -rw-r--r-- 1 marcus marcus 4212 2008-05-05 00:26 Makefile
  -rw-r--r-- 1 marcus marcus 5162 2008-05-05 00:22 speedo.mk

* The NSIS installer itself is also rather simple.  We use stock NSIS
  functionality to achieve almost all features.  Each package has its
  own install and uninstall files, and most are just a list of files
  and directories to install from the output of the autobuilder.
  Although the installer source is very long and packed with rules for
  detecting old gpg installations and such, I can not think of
  anything in it that is actually complicated.

Maybe we have a misunderstand.  Complexity comes, in my view, from
unseparable interacting components.  In gpg4win, most components are
well isolated and function independ of each other.  Even the MSI
package generator has a mostly compatible NSIS parser that is not easy
to break by just extending gpg4win.  Gpg4win is a decidedly linear
program, where one step follows directly from the next.

In my opinion, most perceived complexity from NSIS comes from the
following problems:

* The NSIS language is uncommon and peculiar.  It is also quite
  restricted.  Some things that should be easy are actually quite
  complicated, and unless you are doing this every day you are
  unlikely to know what +1 means as a branch instruction.  The mental
  model you need to follow how NSIS was designed and how it is
  supposed to be programmed is somewhat complex.

* We lack some automation.  Adding a new package still requires too
  many steps, in my opinion, and needs you to edit too many places in
  the NSIS source.  It's cumbersome, tedious and thus error prone.

  NSIS is not a comfortable programming environment, so we don't use
  NSIS programming to make it easier and automate things.  So one
  solution would be to have a meta-language in which we define our
  packages, and from which the NSIS source code would be generated by
  a perl script or something.  This could also be used to generate the
  MSI installer directly instead of parsing NSIS files.

  But, there are development and maintenance costs associated with
  such a solution.  Every feature we need in supporting installation
  of packages would need to be replicated in this meta-language and
  then expressed programmatically in terms of NSIS.  The way we do it
  now is somewhat more flexible, as we don't need to build and
  maintain a layer of indirection.

* gpg4win is extremely long.  We include 57 packages already!  We have
  lots of little rules to support numerous features, such as
  unattended installation, warnings when old versions of software are
  found, etc.  Every rules is simple by itself, but they all add
  together.  Lack of automation and NSIS oddities make the source code
  less than intuitively readable.

  Due to its size, building gpg4win is sluggish, even if only the
  installer is built, on systems that are a couple of years old.

* Inspecting the result is difficult.  The installer is a windows
  executable, so you need Windows to test it, and it's not easily
  debuggable to my knowledge.  In other words: The NSIS source is
  compiled straight to binary, and there is no immediate layer to
  explore that shows the structure of the resulting installer file.

  Although I must admit I hardly ever had a need for it, because it's
  so simple :)

</technical stuff>

> (Also Kleopatra will get small once we apply optimisation to the build.)

I understand that.  It would also be good if kleopatra would get rid
of the little issues like not following the expected packaging and
needing a bin directory.  But even those don't add much complexity,
they just add more tedious and boring rules to follow.

> Defining large packages with one easy requirement 
> can be done without package management.

Please note that there is not a single application package in gpg4win
that has "one easy requirement".  The usual number is 10-20.  Luckily,
most of these can be duplicated and privately installed and used
without problems.  But this is not true for any component that writes
to the registry or provides services!

Even in the Windows world it is highly unusual to be told to install a
third-party package before you install the application.  The reason is
simple:  Such a requirement will kill off your user base exponentially.

And rightly so, as there is no reason why in 2008 an application can
not come with all the required components or download and install its
dependencies automatically.  This is one thing that Windows
applications consistently get right, even if the actual implementation
is inferior.

Now, with claws, things are a little more flexible because gpg is only
an optional extension.  So, in the case of claws the result will be
that exactly 0% plus margin of error will use cryptography, because
nobody can be bothered with going through the trouble of installing
yet another package.

But in general, if I were responsible for a third party package, I
would insist on including all necessary and useful components in the
installer directly.  I would make a considerable effort to achieve
that goal, including forking off the depended on code base and making
it possible to install it in the application tree.  For example, with
gpg, I would install my own version of gpg and make it use a different
set of registry entries.

There are very strong reasons to go this way.  The only way to defeat
this logic is to make it extremely simple and straightforward to
include the dependency from the official source, transparently to the
user.

> > Picking out claws is not a good idea: There are many technical reasons
> > against splitting it out, given the current installer technology, and
> > the same reasons keep Kleopatra in as well, despite its size and the
> > fact that it is the only component in gpg4win that actually comes with
> > an alternative (GPA).  There is no other free e-mail program in
> > gpg4win right now.
> 
> A gpg4win without GPA and with Claws split out, 
> would be a much more canonical package.

Such a gpg4win package would still contain many components that
actually should be shared with other KDE packages.  Once you consider
that interaction, the lid comes off and the whole thing falls apart.

In fact, I feel very uneasy about the fact that we are shipping
dbus-daemon in gpg4win.  This will definitely screw us later on, but
we have no choice at the moment.

Thanks,
Marcus