[iaufwg] FITS panel recommendations; ready for review?

[Don Wells]

William Pence wrote:
> ..the technical panel .. is 
> recommending an extensive set of changes that will need to be made 
> available for wider public comment (e.g., on the FITSBITS email list and 
> the sci.astro.fits newsgroup) as the first step in the formal approval 
> (and possible revision) process..

I have examined the 'Recommended Changes' document dated 07-18, and find 
that the recommendations are acceptable.  My compliments to the chefs!

-=-=- On escape hatches -=-=-

As I read the document, one big issue caused me to pause and think.   
That is the concept of 'escape hatches' in FITS.  This feature has been 
essential for the success of FITS over the past 28 years.   We have a 
long history of leaving certain 'holes' in the specifications, precisely 
to permit clever people to create new designs.  This recommendation 
document appears to be closing some of those escape hatches, and that 
makes me nervous. 

Item 4 of section 1 recommends that we deprecate the special records 
convention.   The justification given is appropriate.  I agree that the 
Generalized Extensions Agreement "appears to be sufficient to meet most 
future needs".   Indeed, I would go further and say that the burden of 
proof is on anybody who claims that the 1984 Agreement is not able to 
satisfy some astronomical data structure requirement.  The variety of 
conventions that have been built on top of BINTABLE show that it alone 
has enough versatility and functionality to solve an enormous range of 
problems.  Also, if we encounter something that BINTABLE cannot do, it 
is perfectly feasible for us to encapsulate *any* other data format 
within FITS if the other format solves a problem that FITS can't solve, 
by creating extensions with XTENSION='<other_format>' and the other file 
in an NAXIS=1 byte array.   Therefore, although this item makes me 
nervous, I have decided to accept deprecation of special records.

Item 10 says XTENSION type names *must* be registered.   I would prefer 
a strong urging accompanied by the usual justification rather than a 
'must', precisely because XTENSION is our most important escape hatch, 
and I don't want to inhibit experimentation unnecessarily.   However, I 
won't insist on this matter.

Item 21 recommends changes to section 7.3.3.2 that would "remove the 
option to use the heap and the PCOUNT keyword in ways other than 
described in the variable length array section 7.3.5".   This wording 
makes me nervous because tricks with the heap and the gap area are an 
obvious potential escape hatch that we left in the Binary Tables 
Agreement in 1991.  I decided to examine the color-coded differences 
document (very nice!).   I see changes marked that seem to be related to 
this item, but I am unsure which changes accomplish the goal of the 
item.   Perhaps this item in the Recommended Changes document should 
have some text added to clarify this point, or maybe the differences 
document is missing some color-coding. .

While examining the differences document, I encountered two sentences in 
7.3.5 that seem to me to be inconsistent.  The first is "The meaning of 
a negative value for either of these integers [in the array descriptor] 
is not defned by this standard."  The second is "The storage referenced 
by an array descriptor must lie entirely within the heap area; negative 
offsets are not permitted." The first sentence leaves open the option of 
a negative offset (and that is an obvious escape hatch, because the 
integers are definitely *signed*), but the second sentence closes off 
the option.  (The text of the second sentence is present in v2.1, so 
this isn't the change made for item 21; I don't remember when this 
prohibition was introduced; certainly the IAU-FWG could remove it if 
they wish.)  Apparently the second sentence overules the first sentence, 
but the situation makes me nervous.   Negative offsets would address the 
gap area, and this might be the basis for some clever design that I 
cannot imagine at this moment.   However, I see nothing to stop a 
designer from using simple integer fields in the binary table to contain 
pointers that her software would construe as data descriptors for the 
gap area.  I have decided not to request removal of the negative offset 
prohibition, but I do suggest that the apparent conflict of the two 
sentences be reviewed.

-Don


-------------- next part --------------
A non-text attachment was scrubbed...
Name: dwellscho.vcf
Type: text/x-vcard
Size: 197 bytes
Desc: not available
Url : http://listmgr.cv.nrao.edu/pipermail/iaufwg/attachments/20070720/de7f51c5/attachment.vcf