Project 12: Hardware features
Toronto 1978:
=============
Professor Hull presented a paper entitled 'Desirable floating-point arithmetic
and elementary functions for numerical computation'. The paper summarized proposed
specifications for floating point arithmetic and elementary functions. The
approach adopted was at the programming level. He was primarily concerned with
specifications which were simple, complete and flexible. The impact of numerical
specialists on hardware design and computing system construction had been far
from satisfactory in the past. Professor Hull said the ideal outcome of
present discussions would be the emergence of a generally agreed set of
specifications which would eventually be adopted by computer manufacturers.
The principal points of the proposals presented were:
base - 10
precision - for variables; user declarations such as FLOAT (3,12)x
where x would have a 3-digit exponent and a 12-digit
fraction. For computation; precision blocks' such as
BEGIN PREC(3)
- the precision to be exact, not 'at least' as in PL/I.
representation - sign-magnitude, normalised.
arithmetic - properly rounded, left-to-right evaluation (or some
other 'obvious' rule)
basic functions - abs etc available
elementary functions - emphasis on simplicity.
error handling - PL/I-like ON conditions
fixed point - is INTEGER necessary?
Discussion:
Dr. Smith queried the proposed absence of integer variables. Professor Hull replied
that a specific function for integer overflow, for example might be necessary,
He accepted Professor Fosdick's surmise that an optimising compiler would be
allowed to optimise code so long as the effect was numerically identical to
evaluation in the proposed standard way. Dr. Reinsch, in questioning the
pursuit of simplicity, felt that the provision of functions such as log would
give some difficulty. Professor Hull responded by suggesting a simple, general
log function and a special one for particular requirements.
---
A paper by Dr. Reinsch called 'Principles and Preferences for Computer Arithmetic'
was presented. Dr. Reinsch described the background to the paper and acknowledged
the contributions of others. He would concentrate attention on hardware (or
firmware) features rather than software. Originally he had envisaged producing
a grading scheme for various possible features but he was now in favour of
stating desired principles and then examining the consequences of those statements
(e.g. leading to the choice of base).
The principle sections of Dr. Reinsch's paper were:
- introduction; largely based on Professor Dekker's earlier IFIP working
papers
- floating point number system; here the extrema and the
resolution factor were introduced. The system was to be
1) Symmetric
2) Balanced
3) contained
There would also be special operands for indefinite and infinity
results. As for the choice of base, Dr. Reinsch did not feel that any
one choice was superior in all respects to other choices.
basic arithmetic operations; in this section a number of desirable
properties were propounded concerning rounding and basic arithmetic
operations. The importance of comprehensive and correct documentation
describing the implementation of these rules in a particular system
was stressed.
- underflow/overflow; any U/O event must always be indicated to the
programmer who could then respond accordingly.
- relations; these should be exact.
- integer arithmetic; a number of principles were presented.
conversions; input/output transfers and precision conversions should
be exact where possible.
Discussion:
Dr. Battiste welcomed what he regarded as the 'top-down' view of Professor
Hull and the 'bottom-up' approach of Dr. Reinsch. Both approaches were
necessary to achieve desired goals. He referred to a problem of 'wrap-around'
recently encountered in the production of mathematical software. Dr. Reinsch
believed that in suck a case, his minimum requirement was to warn the user and
permit recovery in a reasonable way if possible. Professor Hull detected a
difference in emphasis in the two approaches perhaps because the term 'user'
was being used to refer to different kinds of computer users. Dr. Reinsch's
proposed scheme was supported by Professor Dekker. Another aspect of
computation pointed out by Dr. Battiste was the need of the statistician for a uniform
resolution throughout the whole range. Providing such a facility on discrete-
arithmetic machines would always be a problem, said Dr. Reinsch.
A further presentation on machine arithmetic was presented by Dr. Smith. It
was based on a paper by Kahan, Stone and Coonen that he had recently seen.
The paper described the current state of an IEEE-sponsored activity to propose
manufacturer-independent hardware standards. The current proposals bore some
relationship to equipment about to be released by INTEL though now had evolved
somewhat further apparently. (Copies of the paper were distributed to members).
The main topics discussed in the paper were arithmetic storage, operations,
comparisons and exception handling. Three types of entity in the closed number
system were introduced; normalised numbers, infinites and 'nans' (_not a
number). The paper went into considerable detail as to how these entities
should be manipulated. Computations were assumed to take place in an internal
extended precision units. If the result of a computation overflowed, that
was to be detected when the result was transfered to arithmetic registers
accessible to the user. The recommended number representation was sign-magnitude
and the suggested number composition was a 23-bit binary mantissa with an
8-bit exponent. (Dr. Smith offered this explanation for such a choice; it
arose probably from the advent of the 16-bit bus on mini computers and from
the belief that well-formed arithmetic on 32-bit words was often sufficient).
Of the several rounding options presented, the most strongly recommended was
proper rounding. Two forms of arithmetic would be available, projective and
affine. The mod and sqrt functions were to be implemented by hardware. There
was provision for a guard bit, a rounding bit and a 'sticky' bit. A status bit
would indicate when rounding had taken place (thus permitting exact integer
arithmetic?). Finally, the problem of base conversion on input/output was
addressed. It was claimed that numbers which were input then output or vice
versa would achieve their original value.
Discussion:
Dr. Reinsch commented that the proposals seemed consistent with previous
work by the authors. The Chairman asked members to respect the draft status
of the paper before making external reference to it.
---
The meeting resumed discussion of the arithmetic systems considered earlier
in the day.
Dr. Brown gave what he considered to be a simplified but relevant characterisation
of the three main strands of thinking:
(a) where human-related costs dominated, the tendency was towards ideas
such as those of Professor Hull.
(b) if computer costs were paramount (as in weather modelling)
with specialised architecture such as the Cray-1 emerged.
(c) in most cases, however, a compromise developed between user-convenience
(crudely speaking, approach (a)) and the 'machine-dominated' approach
(b), e.g. as reflected in the Stone, Kahan and Coonen paper.
There was a need to develop standards for all three cases.
The meeting discussed Dr. Brown's characterisation at length. Dr. Reinsch
felt that his proposals addressed topic (c) but considered that there were
extremes other than (a) or (b). Professor Dekker said that it may be necessary
to have separate (or at least graded) hardware specification standards where
options to meet particular circumstances were included. In Dr. Battiste's
opinion, questions of implementation of proposed hardware standards (and of
convincing manufacturers of the wisdom of such standards) had yet to be
addressed.
Commenting on the choice of a base, Dr. Brown considered that Brent's
work had established a strong case for base 2 with proper rounding. He
believed that Professor Hull's proposed exact-precision arithmetic might
be costly to implement on present ranges of computers but he supported the ideas
of user-control over precision, except that he wished integer arithmetic to be
retained. Dr. Brown concluded his informal presentation by suggesting that
the INTEL chip, the Cray computer and Professor Hull's proposals were all
cause for celebration and optimism.
Professor Rice felt that current efforts should be directed largely towards
case (c). Those involved in (a) and (b) realised that they had special
needs which would take longer to fulfil.
The Chairman considered that it might be possible to influence the builders
of special computers. He regarded the approach of Professor Kahan et al,
and those of Dr. Reinsch and Professor Hull as all addressing the same
problem but at different levels. Dr. Brown's ideas added a further dimension
still.
Professor Hull felt that inconvenient computational systems build on poorly
designed hardware had been inflicted on users for too long. He was
happy with the gradual identification of higher and lower goals in terms of
computational requirements.
Dr. Smith expressed support for Professor Hull's aims but said that hardware
designers would require more guidance. Dr. Battiste recalled the history of
machine construction and observed that the boundaries between hardware and
software were becoming increasingly blurred. Professor Dekker and Dr. Reinsch
returned to the question of implementing proposals such as those of Professor
Hull. Two particular problems that they identified were the dynamic evaluation
of exponents and of special functions.
---
The meeting resumed discussion of the issue raised by the presentations of
Dr. Reinsch and Professor Hull.
Professor Rice was keen on publishing the papers given, particularly that of
Dr. Reinsch, soon in order to receive comments from a wider audience. Dr. reinsch
described the background to his paper; the Oak Brook meeting had been of
particular significance. He stressed that his paper was intended to influence
the designers of current hardware systems but not to confuse them with too
much justification of design principles from higher-level software considerations.
Members of the group were asked to convey any further views on the paper as
soon as possible. Dr. Reinsch would then submit the revised document via the
secretary to members for formal consideration as a working group document.
Professor Hull found the two level approach acceptable. If the two were
developed in a compatible way, his proposals at the programming level could
be based on sound arithmetic at the hardware level. Professor Stetter agreed,
saying that the machine-level definition was an essential first step, from
which ideas such as Professor Hull's could be more easily implemented. He
felt that Professor Hull's paper should be referred to compiler writers.
To remove possible contradictions between the two papers, Professor Hull said
he was prepared to study both sets of proposals in conjunction. One
potential problem, for instance, might be the treatment of error traps;
Dr. Reinsch's proposal was necessary but was it sufficient at the higher
level? Dr. Reinsch offered to refer to higher level programming issues in
the introduction to his paper. A comparison of the two proposals and that of
Kahan et al was given by Professor Dekker. Dr. Smith said that the Kahan set
of proposals was a good example of a specialised implementation of ideas like
those of Dr. Reinsch. Dr. Einarsson reminded the meeting of Lhe current status
of the Kahan paper.
The meeting then considered the circulation of the two papers for comment as
discussion documents. Dr. Reid and Professor Fosdick wanted an open approach
during the comment phase. A definite decision on Dr. Reinsch's paper was
required soon (at Baden, perhaps) in Professor Rice's opinion to stand by
chance of influencing hardware designers. Dr. Einarsson considered that TC2
approval of such a paper may take some time precisely because it did affect
hardware. He mentioned the IEEE repository as a possible place to publicise
the draft ideas. Another suggestion was to approach appropriate special
interest groups in ACM. Dr. Reinsch wanted to have a quick response from the
numerical analysis community before actively pursuing the views of hardware
designers. It was suggested that the papers might be given at the Baden
working conference and offered for presentation at the Liverpool software meeting
in Easter 1979. This led to a discussion of the desirability of deviating
front the theme of the working conference.
Dr. Einarsson summarised the outcome of the overall discussion with
Dr. Reinsch's and Professor Hull's papers;
- comments from WG 2.5 members should reach the authors by the beginning
of July
- the two authors were encouraged to publish their papers as discussion
documents in SIGNUM (and other journals if appropriate)
- they would act as stand-by speakers at the working conference (and
their papers might, at the discretion of the editor, be published in
the .proceedings).
Baden 1978:
===========
Dr. Reinsch surveyed the state of his paper. It had been distributed to
interested parties and had not met with any objections. Subsequently it
had been sent for publication in the SIGNUM Newsletter. Professor Dekker
sent a suggestion and an introduction was also added. This paper was mailed
to Dr. C. Lawson. Thereafter two other suggestions had been received, one
from Professor Gear and one from Professor Rice. The latter had reworded
the paper slightly to improve the English and sent this to SIGNUM for the
March issue. Other changes were still possible up to the middle of January,
Professor Rice said. There was some discussion on the sentences describing
the status of the paper within the group since all members did not totally
endorse all the viewpoints expressed therein. Professor Hull, for instance,
thought that the original wording adequately represented the differences between
himself and Professor Reinsch on this subject. He thought it bad, however,
that both should publish independently of one another without referring to
the others work.
The Chairman asked Professor Hull, Dr. Reinsch and Professor Rice to form a
sub-committee on this topic. He also enquired about the current state of
Professor Hull's paper and was told that it had been re-written and a
shortened version appeared in the Pasadena Conference Proceedings. As a
result of this the paper had automatically been sent to I.E.E.E. It had not
been submitted to SIGNUM. It was suggested that Professor Hull's paper appear
in SIGNUM, together with a summary linking together the ideas of Professor Hull
and Professor Reinsch. To this end Professor Hull undertook to see if I.E.E.E.
would object to this and Professor Rice promised to investigate the summary.
It was agreed that Professor Hull and Professor Reinsch should send their
papers to the Chairman of TC-2.
Professor Kahan's Paper
The question was asked by the Chairman, in the light of Professor Kahan's
presentation, whether or not WG 2.5 ought to take a stand in the dispute.
Both Dr. Reinsch and Professor Stetter thought some action mandatory but,
whereas the former thought immediate action premature, the latter thought
it necessary. Dr. Brown thought that WG 2.5 could endorse the features
common to both proposals but Professor Gear pointed out that this would
effectively endorse the DEC proposals. He advised against becoming involved.
Professor Fosdick too expressed concern, observing that the Payne proposals
had not been given the same hearing as Kahan's, and noting that WG 2.5 had
nothing to gain by taking a decision. Dr. Reinsch supported a suggestion by
Dr. Ford to try and order machine arithmetic features from 'most preferred'
to 'least preferred'.
Floating-Point Model
Dr. Lawson described the Pasadena Conference, which had 27 speakers, 95
attendees and lasted 2 1/2 days.
Members of the X3J3 committee had attended to the first morning's
session and Dr. Lawson, Dr. Smith and Dr. Brown all spoke at the meeting. It
was his impression that X3J3 were aiming to define a 'core' for the new
FORTRAN which would be more of a subset of FORTRAN 77 rather than
embracing it.
Dr. Ford spoke about the paper, 'Parameterisation of the Environment for
Transportable Numerical Software' and this provoked some interchanges between
himself and Dr. Brown, who was anxious that his own model be accepted. Dr. Ford
wanted to make some modest revisions along the lines indicated by Dr. Cody and
promised to bring specific details of these changes to the next meeting. This
approach was agreed nem. con.
Professor Hull saw no conflict in proceeding with the parameters paper and the
Brown-Feldman paper in parallel. Dr. Brown was keen however to submit proposals
to X3J3 this year, when he believed the committee would be receptive to these
ideas.
Dr. Reid introduced the 'Intrinsic Functions' paper. It was a small paper with
no major changes from the Toronto meeting. Dr. Brown was unhappy about the
paper since it was similar to the Brown-Feldman paper with references to model
parameters deleted. Pressed by Dr. Reid he said that he would sooner go this
way than not go at all but thought it impossible to adequately define the
functions without reference to the model parameters.
Dr. Reid was unwilling to publish the paper as an individual and there was some
discussion as to how the paper should be introduced. A wording was suggested
by Dr. Einarsson, which was subsequently amended by TC-2, to read:
"The paper has been discussed by IFIP WG 2.5 on Numerical Software
and has been approved by the Working Group, but does not constitute
an official IFIP document".
The content of the document was subject to discussion. Professor Gear wanted
functions to be declared INTRINSIC if they are used as such. Professor Dekker
wanted a dynamic definition of EPSILON. Dr. Reid however was anxious to make
only minor revisions at this stage and to finally clear the matter up. The
changes he proposed were:
Page 2, line -3 insert "and k is some constant (e.g. 10)".
In the footnote replace "real" by "processor". Delete the last line and insert
"No floating-point operation can cause underflow if its result is outside
(-sigma, sigma).
Page 4 - bottom. Delete "F is setexponent ..." and read
X_i = setexponent(X_i, integerexponent(X_i)-K)".
Replace "note that ... permissible exponents"
by "It is unfortunate that two function calls are involved in the loop, but
they are such simple functions that we could expect a compiler to provide in-line
code".
Dr. Brown asked Dr. Reid to consider six functions, not three. Professors
Stetter and Hull thought two papers mutually referencing one another a possibility,
but Dr. Smith said.that X3J3 needed help - one proposal not two. The matter was
to be discussed in smaller groups outside the meeting that evening.
Intrinsic Functions
Professor Hull reported that he and six others met the previous evening and as
a result the gap between Dr. Brown and Dr. Reid narrowed. The latter would
acknowledge the proposal for 3 extra functions and also the different definition
of EPSILON. The revised paper would be distributed in early January and remarks
on it invited. Should the comments be serious these would halt publication.
Dr. Brown gave notice that he intends to submit the Brown-Feldman model for
approval at Novosibirsk. He asked that his silence of the Ford and Reid papers
not be interpreted as approval nor prejudice the motion at Novosibirsk.
Dr. Reid's paper was to be given the same support, and introduced by the same
wording, as Dr. Reinsch's. It was to be sent to X3J3.
Arithmetic Proposal
Professor Gear reported on the findings of a sub-committee meeting the previous
evening. They felt that some action was necessary and thought that this should
be in the form of an ordered list of features that the group would like. The
proposed list was
1) Unbiased rounding, choosing even when equal.
2) 32/64 binary base, with 8/11 bit exponent is acceptable.
3) Traps available for overflow, underflow, division by zero and all other
unusual arithmetic conditions (ones in which the true answer cannot be
represented as a valid number). It is necessary to have the ability to
disable traps.
4) Extended precision and exponent range on a local range is desirable, as
is the existence of denormalised numbers.
5) Other types of rounding are highly desirable but unbiased rounding must
be the default.
6) There should be representations for +/- infinity and NANs.
7) Representations should be in order - infinity < - NAN < 0 < NAN < infinity.
If cost considerations are paramount then ordering, other roundings and extended
precisions could be omitted.
The list was the subject of some debate.
Dr. Brown wanted to leave out the last paragraph but Dr. Ford said that it was
important and showed that the group was aware of practical considerations. He
was supported in this by Dr. Einarsson and Professor Stetter. Dr. Brown thought
that the document should contain a reference to Dr. Reinsch's paper. There was
general agreement in principle that there should be a paper, but the details
had yet to be resolved. This would enable Dr. Einarsson to ask TC-2 for their
support in sending the paper to I.E.E.E. when he attended TC-2 later in the day.
Novosibirsk 1979:
=================
Brown reported that his paper on the floating-point model had been
rewritten (IFIP/WG 2.5 (Novosibirsk-11) 611), that the model has been
accepted as a basis for the arithmetic in the language ADA (Ford
queried whether this was really so), that his proposal for
environment parameters had been revised (IFIP/WG 2.5 (Novosibirsk-10)
610) and that Schryer's test program for checking parameter values has
been completed. He drew attention to the preliminary results given in
an appendix to (IFIP/WG 2.5 (Novosibirsk-10) showing values for
parameters and details of certain apparent anomalies. Fosdick remarked
that this table was particularly useful in that it permits ready grasp
of the efficacy of different manufacturers' arithmetic. Reinsch,
however, remarked that certain other dirty features, such as biased
rounding, may pass unnoticed. Schryer's program may need quite long
runs (e.g. a few minutes on CDC 7600) for results in which one can be
fully confident. Gear remarked that different compilers on the same
machine may need different parameter values.
Harwell 1980:
=============
T.E. Hull: Handling of exceptions and some remarks on the design of
arithmetic units (see IFIP/WG 2.5 (Harwell-43) 743
Hull explained how he felt exceptions should be handled at the
language level and describes his discussions with hardware experts on
the possibilities of designing an efficient decimal unit. The outcome
is that the speed of a decimal unit could be very comparable to a good
binary unit, but the cost might be double. He hopes to arrange a
research project involving the actual construction and experimental
use of such a unit.
Discussion
Curtis: The main difficulty is that we are stuck with
architectures based on fixed words (e.g. 32 bits).
Hull: You can pack three decimals into 10 bits.
Curtis: This may permit a 32-bit word to be used efficiently, but
surely the overheads of packing and unpacking would be substantial.
---
Principles and preferences for computer arithmetic
Rice introduced a draft (IFIP/WG 2.5 (Harwell-14)714) of a paper that
briefly summarizes the work of both Hull and Reinsch. Hull was worried
by the apparent conflict and remarked that hardware designers do not
read SIGNUM newsletters. An outlet to the hardware world is needed.
Fosdick said he thought the paper should relate to the Rice IEEE
standard; the group should not appear to be "mudding the waters".
A subcommittee consisting of Rice (chairman), Hull and Reinsch was
set up to prepare a final version for publication, to be
presented for approval at the next meeting (draft: IFIP/WG 2.5
(Harwell-38)738).
Madison 1982:
=============
The "Karlsruhe approach" to arithmetic
--------------------------------------
Kulisch explained that all he needed in Fortran 8x to
permit the use of his arithmetic was the ability to define
his own operators. Hull asked if he did not need additional
rounding modes, but Kulisch said that these could be written
as assembly routines. Smith asked about Ada; Kulisch said
that the problem here is that only existing operators may be
overloaded, but Smith felt that this could be circumvented
by defining another type with a single component. Gentleman
remarked that simple overloading of existing operators did
not give rise to parsing problems but was not so useful.
Rice asked whether there was any chance of acceptance and
Smith said that he thought there was in view of the core
plus modules problem. Einarsson asked whether Kulisch
should not write a tutorial paper and Kulisch said he was
preparing such a paper.
Hardware features, exception handling and precision control
-----------------------------------------------------------
Hull said he would welcome guidance from members of the group on
features for the arithmetic unit CADAC being built at Toronto (see the
29-page document IFIP/WG 2.5(Madison-35)935), copies of which he would
be happy to supply to anyone interested.
Hull pointed out that the group's terms of reference included agreeing
"from a numerical standpoint on a set of hardware and software
features for a computing system", but that we had not always been very
successful (e.g. the attempts by Rice to reconcile the views of Hull
and Reinsch on desirable hardware). What should the group do now? He
felt it should aim to produce agreed documents about "ideals and
principles°' re i) exception handling and ii) precision control.
On exception handling, Hull thought that the user should be able to
specify the action to be taken following an exception and should be
able to set a result (e.g. on underflow, set to zero). He proposed
that each handler should be associated with a given scope (perhaps as
an internal procedure). He noted that our present preprocessors do not
include error handling.
On precision control, Hull noted that the most important idea is that
of separating the precision of operators from that of variables. He
also noted the automatic change of exponent range with change of
precision and that Kulisch's accurate inner product can be expressed
in his language.
Reid mentioned the current X3J3 activity on exception handling by a
group of Europeans. He thought that it would be very desirable for the
group to be taking an interest in this. Dekker felt that exception
handling and precision control were separate topics and that exception
handling is really needed.
Soederkoeping 1983:
===================
The "Karlsruhe approach" or guaranteed accuracy arithmetic.
-----------------------------------------------------------
As Kulisch was not present, Stetter gave a short report. After
reminding the WG members of the main features of the approach, he
mentioned the following activities (besides presentations at various
places and occasions): A scenario seminar exploring the usability of
the approach in various application programming situations ; the
completion of a Fortran Subroutine library (containing the algebraic
routines and many utilities) to be implemented on a mini with the
arithmetic hardware features; further work on applications to analytic
problems; consideration of the chaining of high accuracy algorithms.
The software tool aspects of the available software were stressed.
It was also recalled that the X3J3 committee, at the meeting that
Kulisch attended, has been in favor of hooks in the language to enable
a module implementing "Karlsruhe" arithmetic to be developed.
Documents:
==========
1. W.S. Brown "A realistic model of floating-point computation," in Mathematical
Software III, J.R. Rice (ed.), Academic Press, NY, 343-360, 1977.
2. C.H. Reinsch "Principles and preferences for computer arithmetic," SIGNUM
Newsletter, Vol. 14 (1), 12-27, 1979.
3. T. E. Hull "Desirable floating-point arithmetic and elementary functions for:
numerical computation," SIGNUM Newsletter, Vol. 14 (1), 96-99, 1979.
4. W.J. Cody, "Towards sensible floating-point arithmetic," COMPCON '80, San
Francisco, 1980.