Atari Binary Transfer Format ("ABTF") Standard Proposal
Atari Binary Transfer Format ("ABTF") Standard Proposal
Russ Wetmore [76703,2010] 30 Nov 1985
Modified 15 Jun 85: Added suggested file naming conventions & fixed errors.
30 Nov 85: Clarified ST-specific parameters
SOME BACKGROUND INFORMATION: THIS PROPOSAL IS BASED ON A SIMILAR ONE MADE BY DENNIS BROTHERS, CALLED "MACBINARY."
(I.E. I lifted whole sections from it. Most, if not all, of the credit should lie with Dennis, and Mike Boich and Martin Haeberli, whose MacTerminal program provided the basis for this standard. Dennis' proposal is available in the Macintosh Users SIG (G PCS-23) if you want to compare them.)
MACBINARY IS QUICKLY BECOMING AN INDUSTRY-WIDE STANDARD, AS OPPOSED TO MERELY A SYSTEM-SPECIFIC ONE. IN MANY WAYS THIS PROPOSAL IS SIMILAR ENOUGH TO MACBINARY FOR POSSIBLE CROSS-COMPATIBILITY BETWEEN ABTF AND FUTURE COMPUTERS' FILE TRANSFER NEEDS.
-----------
The following notes are a proposal for a standard format for binary transfer of arbitrary Atari files via a telecommunication link. It is intended for use both between Ataris running (possibly different) terminal programs which adhere to the standard, and for use in uploading arbitrary Atari files to remote systems (where it is presumed that they will be stored as an exact image of the data transmitted). A proposal is also made for standard processing to be performed on text files transferred via a protocol, to maximize the likelihood that text files transmitted to a remote system will be usable by that system, and that text files received from a remote system will be usable by an Atari user. It is recommended that the format and procedures described here be referred to as "ABTF", and that any terminal program implementing this format and procedures be called "ABTF-Compatible". It is also STRONGLY recommended that files transferred under this standard utilize the extension ".ABT" on hosts to signify usage of the standard. Since the actual filename is stored in the transmitted header, the downloader can resolve the proper filename without having to intervene in the process. The binary format described is independent of the communication protocol used to accomplish the transfer, and assumes only that an 8-bit transparent transfer can be accomplished.
THIS IS AN IMPORTANT POINT! THIS FORMAT IS NOT A PROTOCOL, BUT MERELY A METHOD OF FORMATTING A FILE SO THAT IT CAN BE LOGICALLY RECONSTRUCTED WHEN DOWNLOADED!
Such protocols as Christensen (usually referred to as XMODEM or MODEM7), Kermit, and CompuServe A or B meet this requirement. Because of the proposed standard's XMODEM heritage, there is a requirement that the transmitted data be padded (with nulls) to a 128-byte boundary at certain points, but this in no way implies that a block-oriented protocol must be used. The basic format proposed is similar to that used by Apple's MacTerminal program, written by Mike Boich and Martin Haeberli. In brief, the binary format consists of a 128-byte header containing all the information necessary to reproduce the file's directory entry on the receiving Atari; followed by the file's data (padded, if necessary, to a multiple of 128 bytes). The length of the data is contained in the header. The format of the 128-byte header is as follows (offsets and lengths are given in decimal):
Offset Length Contents ------ ------
-----------------------------------------------
000 1 Zero. This is a "version" byte.
001 1 Length of filename.
002 63 Filename (only "length" bytes are significant).
(Note: bytes beyond the physical limit of
the filename size for a particular computer's OS
are NOT guaranteed to be any particular value.)
For compatibility and integrity purposes, only the
filename itself (sans device and subdirectory
specifiers) should be included. It is up to the
receiving terminal program to assign the desired
device path.
065 8 Non-guaranteed or system-specific information.
073 1 File attributes. The only attribute supported on
8-bit versions would be whether or not a file is
locked - bit 0 is 1 if locked, 0 if not. On ST
systems, this byte is the one returned by the
Fattrib() GEMDOS call:
bit 7 - (not used)
bit 6 - (not used)
bit 5 - Closed
bit 4 - Subdirectory label
bit 3 - Volume label
bit 2 - System file
bit 1 - Hidden
bit 0 - Read only
074 1 GUARANTEED Zero.
075 7 Non-guaranteed or system specific disk information.
082 1 GUARANTEED Zero.
083 4 Data length (bytes).
087 4 Non-guaranteed or system-specific information.
091 8 File's creation date. (For the ST, and those 8-bit
DOSes that support a real-time clock, like
SpartaDOS. Zero filled if not supported.
Time is represented in the following format:
091 1 Day (1 to 31)
092 1 Month (1 to 12)
093 1 Year (0 to 127: years since 1980)
094 1 Hours (0 to 23: 24 hour format)
095 1 Minutes (0 to 59)
096 1 Seconds (0 to 59)
097 2 Reserved
Note that this could easily be altered to support
TWO dates by utilizing 4 bytes each to represent
each date, using the ST GEMDOS format for time/date
099 25 Zero fill (reserved for expansion of standard).
124 2 CRC of the first 124 bytes of the header block,
for integrity purposes.
126 1 Computer type. Proposed to be $01 for Atari
computers. ($00 currently refers to a Macintosh.)
127 1 Computer OS ID. Proposed to be $00 for Atari 8-bit
computers, and $01 for ST's. ($00 currently refers
to a Macintosh.)
Note that it is the responsibility of the receiving terminal program to resolve file name conflicts, generally by somehow modifying the name of received file if there already exists a file with the original name on the target disk. It is suggested that Atari-based terminal programs implement two modes of protocol transfer: text and binary. Text mode is used for unformatted files of ASCII text (i.e. no control characters except TAB, FF, CR and LF; or "inverse" characters, meaning characters with the high bit set), and binary mode (using the binary format proposed here) is used for all other files. Binary mode may also be used on text files, of course, if it is desired to preserve such things as the file's creation date and attributes. The intent of text mode is to provide compatibility with text files on other systems. Towards that end, it is recommended that a linefeed be inserted after each return character as the text file is transmitted, and that, in the case of block-oriented protocols, the last block be explicitly padded with zeros if the text does not end on a block boundary. When receiving in text mode, linefeeds and trailing zeros should be stripped. If a CTRL-Z (Hex 1A) character is received following all other text (and before any null padding), it should also be stripped (Ctrl-Z is commonly used as a text terminator on GEMDOS and some other systems). Note that the above discussion applies only to text files transferred under some protocol, where an exact image of the transmitted data will be stored in a file on the remote system. When receiving a file via a protocol, a terminal program may distinguish between ABTF-formatted, and non-standard or text modes quickly by examining bytes 0, 74, and 82 of the first 128 bytes received. If they are each zero (and at least 128 bytes have been received), then an integrity check should be made using the CRC value in bytes 124 and 125. This CRC value would be a standard 16-bit cyclic redundancy checksum (too technical to describe here) calculated using the first 124 bytes of the header block. If the CRC value is determined to be correct, then it is very safe (i.e. 99.99..% assured) to assume that an ABTF-formatted file is being received. Terminal programs implementing possible future versions of the proposed standard would, of course, accept an appropriate set of version numbers in byte 0. Note also that compatible extensions of Version 0 of the proposed standard are possible that involve transmission of additional information following the information described here. For this reason, a terminal program should be implemented so as to perform the proper receive procedures for all data sent to it, but to ignore any data that it does not know what to do with. Since a text-mode document does not contain a file name, it is suggested that when text-mode is detected, a file be opened under a dummy name on the receiving Atari, the text written to that file, and the file renamed to a name selected by the user after the reception is completed. This will avoid problems caused by indeterminate delays for name selection at the beginning of the file transfer. It is desirable to allow the user to specify the destination disk drive in advance of the actual start of transfer for either type of transfer. Two methods are suggested for this: provide a "Select Destination Drive" menu selection, presumably in the menu containing the "Receive File" selection; or query the user immediately after the "Receive File" menu selection is made. Either or both of these methods could be implemented in a given terminal program - the independent "Select Receive Drive" method is particularly desirable if an automatic receive facility is implemented. Note that this proposed standard lends itself VERY well to such a facility. (See Addendum A for a suggested procedure for implementing batch transfers.) A short history of Atari-based terminal programs and why we need ABTF
----------
In the beginning (to coin a phrase) there were no terminal programs available for Atari 8-bit computer owners. Jim Steinbrecher of M.A.C.E. developed what has become the basis for most all Atari terminal programs to date: his AMODEM terminal and AMIS BBS programs. Without getting into technical details, there are several conventions of Atari's DOSes that make it fundamentally impossible to use the XMODEM transfer standard without some modifications. Jim, William Volk, and others worked out a kludge that persists to this day. Unfortunately, the method they worked out has a couple of problems of its own, most notable of which is incompability between Atari 8-bit terminal programs and foreign computer systems (like the Atari ST.) The beauty of this simple extension is that it makes transferred files transparently compatible between any computer systems, no matter what downloading protocol is being used. For systems such as CompuServe, Delphi and People Link; which don't support this standard; an image of an uploaded file is maintained which a terminal program (which DOES support the standard) can interpret. Note that since the protocol being used is NOT important, a file uploaded using XMODEM protocol could be downloaded using CompuServe A- or B-protocol, or whatever protocol system the user's terminal program is capable of handling, within the constraints of this proposed standard. Also, I assume that many Atari sysops, for example, will be "upgrading" their BBSes to ST model computers, but will want to continue to support the old 8-bit computers in their download libraries - this proposal would allow files for any type computer to be stored a host, no matter what the host supports. It is assumed that patches to AMODEM and other popular public domain and commercial terminal programs to support this standard would be made available in short order. Note that some modem manufacturers (like Hayes) have indicated that they are modifying their own terminal programs to support versions of this transfer standard. Please address comments or questions on the above proposals to:
Russ Wetmore
Star Systems Software, Inc.
80 Chaney Drive
Casselberry, FL 32707
CompuServe I.D.: 76703,2010
Delphi I.D.: RUSSW
PLINK I.D.: ATARI*RUSS
(Final note: Atari 8-bit HomeTerm will be supporting this standard (or a similar one) in its next update, and ST HomeTerm will feature it on release. I have spoken with other terminal program authors who have expressed tentative support of this standard in their programs. Also, during the changeover process, until more terminal programs supported this standard, a SIMPLE utility would be required to translate ABTF-formatted programs by non-supported terminal programs. I stress the word SIMPLE. The actual translation process is minimal, and requires nothing more than fetching the length of the data after the header, skipping over the header, and reading/writing the given length to the target file.)
Addendum A ---------- First proposed extension to ABTF
It is proposed that byte 99 of the header block be considered a "block transfer" flag. A value of 0 would mean that only this one file is being sent. A non-zero value in byte 99 would mean that another file follows this one. Each file to be sent would have its own header block. This process would continue ad infinitum until a header is received with a zero in byte 99 (meaning no more files follow after completing the download of this one.) Note that folder/file relationships could be maintained - a subdirectory can be considered a file of zero length with the subdirectory file attribute bit set. Another possible extension here would include a provision for "selecting" a subdirectory, previously created, in order to properly ascertain parent/child subdirectory relationships. Possible extensions to ABTF contingent on technical details
------------
On the ST, document type classification is desired in order for an application to automatically open when an associated document is "opened." This would mean directly manipulating the DESKTOP.INF file on the target disk, whose location cannot be determined automatically. Also, unless an application is specifically extended with .TOS or .TTP extenders, the DESKTOP.INF file might also be desired to be manipulated to simulate the "Install Application" function of the Desktop program. GEMDOS has a "load and execute process" function. This function allows programs to call other programs, with control returning to the parent process on termination of the child process. A provision could be made to directly execute a just-downloaded file to perform specific tasks, such as display graphics pictures (whose data would immediately follow), specially handle specific BBS features (such as "foreign" downloading protocols not understood by the terminal program), etc. A possible, but highly recommended against, extension would involve alerting the host terminal program to comprehend the "8-bit Atari XMODEM kludge" for special ST<->Atari 8-bit computer situations.
