Project 8: Criteria for evaluating quadrature, ODE and
integral equation software
Interpreted as covered by the Baden conference on performance
evaluation, see project 18.
-----------------------------------------------------------------------
Oxford 1975:
============
Special session on evaluation of algorithms summarised in the minutes, pp 4-6.
Position Paper: Building a Library of Numerical Algorithms: A Case Study
from the Handbook on 'Linear Algebra' - C H Reinsch
The author observed that there are there stages in the production of
mathematical software: (i) the invention and selection of mathematical
constructions (methods of computation), (ii) their casting into
numerical algorithms (uniquely determined sequences of basic operations),
(iii) the final transition into programs working on an individual computer
(e.g. a FORTRAN program for an IBM 370 machine).
He felt that the first stage is the domain of individual research so
that a group devoted to communication could contribute little in this
field. Likewise, he stated his belief that much of the third stage is
too computer-specific to be the target of a group activity, On the other
hand, it was the author's view that the second point would yield a
more natural objective for the group.
The working group would compile from the literature recommended methods
of computation and evolve each of them into the form of a numerical
algorithm as defined above and so gradually build up a universal library
of such algorithms.
The paper then discussed various premises of such an enterprise based
on the experience gained in the work on the Springer Handbook of
Automatic Computation, Volume II: Linear Algebra (eds. J. H. Wilkinson
and C. H. Reinsch).
Discussion:
There was agreement with Dr Wilkinson that the divisions in the papers
in the Handbook were a good idea.
The respective merits of Algol 60 and FORTRAN were argued and Dr Lawson
remarked that in some ways languages such as BABEL represented a gradual
drawing together of opposing viewpoints. Pursuing Dr Reinsch's
theme that algorithms should be written in an Algol 60 subset to enable
them to be translated easily into other languages, especially FORTRAN,
Dr Smith enquired how dynamic arrays could be translated. He felt
that it was wrong to expect people, unfamiliar with the algorithm, to be
able to deal adequately with difficulties that may arise in different
machine implementations.
A theme, which was to recur throughout the afternoon, was introduced by
Professor Stetter who wondered whether the group was primarily concerned
with software or algorithms. He and Dr Lawson favoured the Handbook
approach, and welcomed EISPACK, which sets standards for others to
follow in the creation of libraries. Professor Stetter emphasised
the need for means of evaluating an algorithm and mentioned the work of
Professor Hull's group for ordinary differential equations.
At this point the Chairman conveyed a message from Professor Hull which
explained his absence from the meeting because of sickness.
Professor Hull had volunteered to -
(i) Co-ordinate the creation of a set of test problems for stiff and
non-stiff problems in ordinary differential equations and make
them available on magnetic tape.
(ii) Attempt to categorise these problems.
(iii) Supplement each problem with a routine capable of solving the
problem within specified requirements.
The meeting adopted Dr Ford's suggestion that he should write to
Professor Hull expressing regret at his absence due to sickness.
The group then discussed possible objectives. Dr Reid felt that the
creation of a collection of a set of machine readable test problems was
most important and he was supported in this by Dr Ford. Professor Stetter
suggested that the group consider the qualities required of a good algorithm.
He felt that a survey of user requirements would be useful and observed
that a user should not have to choose between algorithms, but have this
done automatically for him by the computer, given details of his
particular problem.
Discussion of the criteria for evaluation of numerical software ranged
through the choice of algorithm, performance, documentation and user
convenience of software, the tools for the evaluation of software and
the collection of statistics on user needs. Professor Stetter felt
that more progress would be made by concentrating the Group's attention
on specific subject areas and quadrature, ordinary differential equations
and integral equations were mentioned as candidates for consideration.
Of them quadrature appeared most favourable as a first step.
Amsterdam 1977:
===============
Dr Smith and Professor Stetter are separately working on this and
the conference next year will address this problem.