Apple II History Part 15: PRODOS & Beyond

@SteveJobs

Published in

Apple II History

· 5 years ago

Compiled and written by Steven Weyhrich
(C) Copyright 1992, Zonker Software

(PART 15 -- DOS 3.3, PRODOS & BEYOND) [v1.2 :: 30 Sep 92]

DOS 3.3

In August of 1980, Apple released an upgrade for DOS, to version 3.3. This upgrade was an important one. It consisted of not only a new System Master disk, but a hardware upgrade chip as well. The original disk drive had been designed with the ability to read and write 35 tracks of 13 sectors each on a 5.25 inch disk. At 256 bytes possible per sector, this made the disk capable of holding 113.75K of information. Since it was designed to have DOS present on each disk in the first three tracks, and the catalog took up another entire track, there was actually only 100.75K available for data storage. Steve Wozniak, the author of the original DOS disk driver (RWTS), had found a way to increase the storage capacity of Apple floppy disks. Changing slightly the method used for encoding data on the disk made it possible to have 16 sectors per track, instead of the original 13 sectors per track in DOS 3.1 and 3.2. This resulted in a disk that could now hold a maximum of 140K of data (124K excluding DOS and the catalog track), a 23 percent increase over the 13 sector disks. The remarkable thing about this upgrade was that the disk drives themselves did not need to be changed to make this possible. Only the ROM program on the Disk II controller card needed to be changed to make the move to DOS 3.3. Those users who bought this upgrade to DOS 3.3 had to change the ROM chip on the disk controller (or have their dealer do it for them). An updated and greatly expanded version of the DOS manual was also included in the DOS 3.3 upgrade.<1>

DOS 3.3 - FEATURES

The DOS 3.3 System Master disk included many programs that had previously been present on the DOS 3.2 Master, plus a few others. The "COPY" program (used to copy entire disks) was translated to Applesoft as "COPYA" for those II Plus users who didn't have access to Integer BASIC. The newer COPY programs also worked properly on single drive systems (previously, you had to have two disk drives in order to use this program to copy a disk). To allow users to startup their older 13 sector DOS 3.2 disks, a binary program called "BOOT13" was included. (Also, a separate disk called "BASICS" was included that could be used in the same way as a pre-boot for 13 sector disks).<1> Because of the changes in the ROM controller, it was not easy to read disks formatted under DOS 3.2 directly from DOS 3.3. It could have been incorporated into DOS 3.3, but would have called for a major effort in rewriting the track and sector access routines, as well as making DOS larger than the earlier versions. Instead, Apple supplied on the System Master disk a conversion program called "MUFFIN" to allow files to be moved from 13 sector to 16 sector disks. Enterprising hackers in the Apple II world made modifications to MUFFIN and created DE-MUFFIN, a DOS 3.2 utility to convert the files back to the 13 sector format.<1> Rich Williams at Apple wrote the MUFFIN program (which was supposed to stand for Move Utility For Files In NewDOS).

The System Master disk also contained a new utility called "FID" (which started at version "M"; just like DOS "3", nobody knows why the first public release didn't start with "A"). FID, written entirely in assembly language, allowed easier copying of files, particularly Text and Binary files that couldn't simply be LOADed and SAVEd from one disk to another, as could Applesoft and Integer programs. The name "FID" was odd, however. The Apple manuals said it stood for FIle Developer, but Rich Williams (who also wrote this utility) said that the original name of the program was FISHEAD (which had some sort of mnemonic meaning that he could no longer recall). Apple Marketing said he couldn't name a program FISHEAD, so he changed it to FID, which they said was okay. It really stood for Fishead In Disguise (or Fishead In Drag by some within Apple).<2>,<3>,<4>

Some Apple II users didn't like to have to use utility programs to manage their collections of disks in both the 13 and 16 sector formats. One method that was used to overcome this inconvenience was to piggyback the old and the new disk controller ROMs and use a switch to toggle between systems. The most elegant solution I've found was a ROM chip that plugged into a special card (the ROMPlus made by Mountain Hardware, or the ROMBoard made by Andromeda). A call to a memory location would switch between DOS 3.2 and 3.3, making file conversions quite easy. Soft Ctrl Systems, the company that sold this Dual DOS ROM also sold ROMs that gave instant access to an Applesoft renumber and merge program, an Applesoft editor, and a specialized disk command menu and disk map.<2>

Another change found on the DOS 3.3 System Master was in the method used to load the alternate BASIC. Since by this time the Language Card was available (which, as you should recall, was simply 16K more RAM to add in parallel to the Apple II ROM), there were two groups of users to service on bootup. For Apple II Plus owners, there was a file named "INTBASIC", which would load Integer BASIC onto the Language Card. For the older Apple II (non-Plus) users, the file "FPBASIC" would be loaded onto the Language Card when the DOS 3.3 disk was booted, making Applesoft available. The last version of the DOS 3.3 Master disk, released with the Apple IIe, used a new utility to load these files which was significantly faster than the standard DOS BLOAD command.

DOS 3.3 - MISCELLANEOUS

A rumor expressed in a letter to Call-A.P.P.L.E. magazine in January 1982 suggested that up until Christmas of 1980 there never had been an assembly language source listing of DOS. The writer of the letter stated that changes made to DOS up until that time were done by patching it with the mini-assembler in the Monitor. However, during a phone interview in September 1991 with John Arkley at Apple, he said there always was a source code listing for DOS, as far back as DOS 3. He believes the writer of the letter may have been referring to the problem with the lost Autostart ROM source code (see Part 6 of this History). Arkley stated that the earliest versions of DOS were written using a cross-assembler on a Horizon microcomputer.<5>,<6> He also said that the only part of DOS 3.3 that was assembled from scratch was the new RWTS. The rest was merely attached to RWTS and "conditionally" assembled (a programmer's term; sorry). They made a few patches to fix bugs in the File Manager and Main DOS routines, but did so only in very specific places, to avoid moving undocumented entry points that programmers had been using up to that point.<3>,<4>,<7>,<8>

DOS 3.3 - LIMITATIONS

The major limit of DOS 3.3 was that it, like its predecessors, was designed specifically to support the Disk II drive. Hard disks, RAM disks, and 3.5 disks (like those used in the Macintosh when it was released in 1984) could not be directly used with DOS 3.3.<9>

PASCAL SYSTEM

The Pascal system was released in 1979, prior to the DOS 3.3 upgrade. It used the same hardware upgrade to the Disk II controller as was included with DOS 3.3. The method used by the Pascal disk system to store files was quite different from that used by DOS, however. Instead of the 256-byte "sectors" used with DOS 3.2 (and by 3.3), the Pascal system used 512-byte "blocks", using two sectors per block. Pascal used the larger 140K disks from the beginning, and its method of file naming was somewhat more limited. Instead of names that could be as long as 30 characters and could contain any ASCII character (as was the case with DOS 3.2 and 3.3), Pascal files could be only 15 characters long, and could contain only letters, numbers, or a period. It was designed with a little more flexibility in the types of files that could be created, however. Instead of DOS 3.2's limit of eight different file types ("A", "I", "B", "T", and the other four little used ones), Pascal was designed to allow many more, and used a two-byte code to designate file types. A Pascal file entry also had space for a date when the file was created or updated. DOS 3.2 or 3.3 could not easily do this, even if a clock card was installed.<7>,<10>

Pascal disks differed also in being able to have a unique name to designate each disk. DOS 3.2 and 3.3 could be formatted to use up to 254 different volume "numbers", but this feature was seldom used and did not allow disks to be very unique. The Pascal disk name could be up to 7 characters in length, and had the same limits of character choice as did file names. Another feature of the Pascal disks that differed from the older DOS disks was how space was allocated on a disk for a particular file. Under DOS 3.2 and 3.3, space was used on the disk to identify which sectors were used and which were free. When a new file was created or an existing file was enlarged, this track/sector list was consulted by DOS to find where free space could be found, and the list was updated when a new sector was used. The advantage was that all space on the disk could be used as it was needed, but the disadvantage was that a file could be "fragmented", with the sectors that made up that file scattered throughout the disk.

Pascal disks did not have any map of free blocks. Instead, a Pascal file used only consecutive blocks on a disk, and a new file would be started following the end of the last file on the disk. The advantage of this system was faster access to disk files, since they were all on one continuous piece of the disk. The disadvantage was that if a file was deleted, the newly freed space could not be used unless Pascal's "Krunch" utility was used to move all files forward over the unused space.

The Pascal system also included some other built-in disk utilities, an assembler, and a compiler. As part of this system one could also purchase from Apple a compiler for FORTRAN programs and a few other computer languages.<10>

CP/M

With the release of the Microsoft CP/M Softcard, a disk system was needed to handle this foreign programming environment. (Recall from Part 12 of the History that the CP/M system gave Apple II users a Z-80-based computer inside their 6502 computer and, therefore, access to programs and utilities that were previously unavailable). CP/M disks were designed to use four 256-byte sectors as one "block" (twice as large as the Pascal "block"). Like DOS 3.2 and 3.3, the first three tracks on the disk were used for the CP/M operating system which was loaded into memory when booting the disk. Like Pascal, the CP/M directory was found at the start of the disk, instead of in the middle as DOS was designed.

Apple II CP/M disks followed the standard CP/M file naming system. A file name consisted of 8 characters, followed by a period, and then a three character "extension". One interesting feature of CP/M files was that if a file was longer than 16 CP/M blocks (64 DOS sectors), a new directory entry would be made with the same file name. This entry had an extra byte set to show that this was a continuation of a previous file, instead of a new, separate file.<10>

SOS/PRODOS

The operating system designed for the Apple III computer was called "SOS". This title arose from the Apple III's code name, "Sara", which itself came from the name of engineer Disk Huston's daughter. Originally, then, SOS stood for "Sara's Operating System". The manuals released with the computer, however, used the more professional-sounding name "Sophisticated Operating System." SOS was the first operating system for a microcomputer to use the concept of "device drivers", which were programs taken from the startup disk and made part of the operating system. These drivers told the computer how to communicate with the various devices that were attached to it, from a variety of disk drives to the keyboard and monitor. This gave flexibility to the Apple III to use new technology as it became available.<9>

When Apple designed the Apple III, they were under constraints of maintaining some compatibility with the Apple II disk format. They used the same disk controller and the same capacity disks as the Pascal/DOS 3.3 systems: 35 tracks, of 16 sectors each. However, the engineers were free to make any changes they wanted in the way in which files were stored on the disk. They came up with something that was a hybrid between the DOS 3.3 and Pascal methods of file storage. From Pascal they took the concept of using 512-byte blocks as the basic unit of storage, a two-block "system loader" program at the start of the disk (this loader would locate a larger system file elsewhere on the disk to actually start the operating system), and a four-block main catalog (which they called a "directory"). From DOS 3.3 they used the concept of disk maps and block lists for each file, allowing parts of files to be stored anywhere on the disk (and eliminating the need for the Pascal "Krunch" function). The SOS filing system also continued the use of a byte to identify different filetypes, space for a date (and time) of file storage, and the 15 character file names using only letters, numbers, and a period. Because the Apple III was intended to be a business machine and had to be able to access larger disk devices than were allowed for the Apple II, they also added the ability to create and use different levels of file directories. A single four-block directory had space only for 51 files; even if it was enlarged to allow more files, on a large disk it would soon be difficult to find a file in a list that got longer than a couple of hundred names. The SOS disk file system also would allow files to be as large as 16 MB, and a single disk volume could be up to 32 MB in size. In 1981, when the 5 MB Profile hard disk was released by Apple for the III, this limit of 32 MB was considered to be more than adequate.

In 1984, when ProDOS was released for the Apple II as a "Professional Disk Operating System", the same file structure was used. In fact, the disks were so designed that a disk created by the Apple II ProDOS formatter installed an Apple III SOS loader segment in the second block on the disk. This made it possible to boot the same disk on either an Apple II or an Apple III, if the necessary system files unique to each computer were present on the disk. Also, files could be shared easily between the two computers. Even as late as 1992, when the Apple III has been out of production for eight years, disk formatted by Apple II System Utilities still have SOS boot information located on block 1. What may be even more amazing is that this disk system for the Apple III, released in 1980 (and probably designed in 1978 or 1979), is still flexible enough to be useful for Apple II's in 1992.<10>

PRODOS

The original DOS for the Apple II was designed primarily to support BASIC. If a programmer wanted to make use of the disk system for an assembly language program, he had to make use of undocumented, low level calls to the DOS File Manager, or possibly to some of the Main DOS Routines. This method was clumsy, and often made inefficient use of memory, as DOS expected that any calls made to it were done on behalf of BASIC. Moreover, this tied the hands of programmers at Apple in their ability to enhance DOS, since any changes they might make would most likely change internal addresses, and cause older software to malfunction if used with the revised DOS.

Another problem with DOS was speed. Since each byte read from the disk was copied between memory buffers three times, much of the disk access time was spent in moving things around in memory. Consequently, as hackers took DOS apart and found better ways to do things, several variations of DOS speed-up programs appeared by 1983, including Diversi-DOS, ProntoDOS, and David-DOS. Each of these programs were mutually incompatible in terms of the low-level calls they made, and had slightly different ways of speeding up DOS.

DOS was also limited since it was device dependent. It was designed to work quite well with the Disk II drive, but to make use of a hard disk or RAM disk (a pseudo-disk "drive" that was actually RAM memory, had no moving parts, and was therefore quite fast), DOS had to be patched. This usually made it impossible to use different brands of hard disks together, or to use a hard disk and a RAM disk simultaneously.

Other problems with DOS included poor support for interrupt signals generated by various hardware devices, obstacles in designating memory areas as protected from being overwritten by DOS, and the difficulty in customizing DOS for special functions.

With the introduction of ProDOS, all of these weaknesses were addressed. ProDOS would run up to eight times faster than DOS in accessing 5.25 disks. It supported a standardized protocol for hardware-based devices, allowing reads, writes, status calls, and formatting (erasing). This allowed a large variety of disk devices to be used on an Apple II. Support was also included for a hardware clock, allowing date- and time-stamping of files. Hardware interrupts were supported, necessary system calls were placed in a standard location in memory (called a "global page"), and memory could be protected from being overwritten by the actions of ProDOS.

Because the functionality of this disk operating system was enhanced so much, its size grew as well. To specifically support Applesoft BASIC, a separate "SYSTEM" program was included that worked nearly the same as the older DOS 3.3 did. In addition, it included some further enhancements that had been requested for years by Applesoft programmers. The only disadvantage of the new ProDOS was that it did not support Apple's original Integer BASIC, since the ProDOS program loaded itself into high memory where Integer BASIC was loaded in an Apple II Plus. Since very little development of software had been done in Integer BASIC since the introduction of Applesoft, this was felt to be a reasonable trade-off. And if Integer BASIC was needed, it could still be run under DOS 3.3. At the time of this writing, there has been no release of a ProDOS system program that would support Integer BASIC (with the exception of an Integer BASIC compiler distributed by ByteWorks in late 1991 for instructional purposes).<1>

PRODOS 16

When Apple released the IIGS, with its considerably greater power compared to the older 8-bit Apple II's, changes were needed in the operating system to better manage that power. This had to be done with another goal, that of maintaining compatibility with older Apple II software. The new operating system was called ProDOS 16, and the operating system intended for use with 8-bit software (both on the IIGS and on the older Apple II's) was renamed ProDOS 8. But ProDOS 16 version 1.0 was somewhat of a temporary fix to the problem of disk access for 16-bit software. It was not written in 16-bit code, and it simply translated the new system calls defined for ProDOS 16 into ProDOS 8 calls to actually carry out disk activities. As such, it was slow and cumbersome.<9>

GS/OS

With the experience of SOS, ProDOS, and the Macintosh Operating System to draw from, Apple engineers and programmers devised a yet more powerful and flexible disk operating system for the Apple IIGS. Written completely in 16-bit code, GS/OS was released in September 1988. It was more than a disk operating system, but a truly comprehensive operating system that also handled keyboard input, monitor output (text and graphics), mouse input, printers, modems, and more. In these respects it was just as powerful as the older SOS written for the Apple III back in 1980. But they also added a new concept.

Although GS/OS would allow an Apple IIGS to communicate with disk devices that had not been used on an Apple II before, there would still be the limits of having to know exactly how files were stored on that disk. ProDOS could only handle files stored in the specifically defined ProDOS/SOS format; DOS 3.3 could only handle files stored in that format; and so on. To make this new system as broad-based as possible, Apple programmers built into it the concept of a File System Translator (FST). With the appropriate FST teamed up with a suitable disk driver, GS/OS could theoretically be able to read any disk created by any computer. The FST simply translated the requests made by GS/OS into the language "spoken" by the disk it was trying to read. This task had never before been attempted by a computer company in designing a disk operating system. Apple, recognizing that the computers used in the real world would never be 100 percent Apple, made it possible to simplify transfer of data between different computers. The concept was first implemented in a limited fashion on the Macintosh, when the Apple File Exchange program was modified to be able to use MS-DOS disks. The Mac system is now also able to add its equivalent of an FST for the ProDOS and MS-DOS disk systems, but not as easily as has been implemented in GS/OS.

GS/OS was also made more flexible by removing the older Apple II method of identifying a disk by the slot where its disk controller was attached, and removing the limitation of only two disk devices per slot. The limits of maximum file and disk size built into ProDOS 8 were expanded. A GS/OS file or disk volume can be as large as 4 GB (gigabytes), or 4096 MB to be more specific. However, when GS/OS is dealing with ProDOS disk volumes, it still has to stay within the limits of ProDOS (files no bigger than 16 MB, and disk volumes no bigger than 32 MB).<9>

System Software 5.0 for the IIGS was introduced in May 1989. It added speed, speed, and more speed to many features of the IIGS, accomplishing this through more efficient software coding. There were patches to the IIGS ROM Toolbox to improve throughput in many of the built-in capabilities of the machine. A new feature called "Expressload" was added, making it possible for certain program files to load from disk up to eight times faster. GS/OS was modified to be capable of staying in memory during a switch to ProDOS 8 applications, making the return to GS/OS significantly faster. The text-based control panel was supplemented by a new graphics-based one that was accessible in the same way as other 16-bit desk accessories. Access to 3.5 disks was accelerated by implementing a feature called "scatter read", which could take an entire track (rather than just a single block) of data from the disk at a time. An FST for AppleShare was added, allowing a IIGS attached to an AppleTalk network to access the file server as a disk. It also included an FST to allow access to CD-ROM drives, new utilities for disk partitioning, and it had an intelligent "Installer" program to make it easier to install system or application files.<11>,<12>

Because of further improvements in features, System Software 5.0.2 (an upgrade to 5.0) required a minimum of 512K memory, and worked best with 768K or more. Versions 5.0.3 and 5.0.4 needed a full megabyte of memory.<9> An improved "standard file dialog" was included in the system tools for 5.0.3, (making it possible to choose files more easily for loading into an application), as were improved drivers for the ImageWriter II and ImageWriter LQ printers. System 5.0.4 was released six weeks after 5.0.3 to fix some remaining important bugs discovered too late.<12>

GS/OS SYSTEM 6

Before System 5.0 was released, plans were already in store for further improvements to the system software. Apple IIGS "power" users were calling for the ability to use Macintosh HFS (Hierarchical Filing System) disks, as well as the older Apple II DOS 3.3 and Pascal formats. Although there were some simple third-party translation programs available that allowed transfer of files from Mac disks to ProDOS disks, they did not provide the same ease of use as did the direct access possible with ProDOS and CD-ROM files. Although it sounded to these users like a relatively straightforward proposition, the increased complexity of the Mac HFS directory structure complicated things. Not only did the Mac disks contain more information about each file than did ProDOS disks, but the names of files on Mac disks (as on DOS 3.3 disks) could contain characters that were not "legal" for ProDOS file names. To help with this problem, the new FSTs were designed to watch for potentially illegal filenames, and to make suggestions for alternate names that were legal.

Apple software engineers had always made it clear to programmers clamoring for additional FSTs that such changes were more than just dropping the new FST into the System/FST folder on a boot disk. Modifications were necessary throughout GS/OS to accommodate these new features, and the time needed to make these changes was becoming longer than originally planned. To allow some improvements to be made available without waiting for them all, the system software engineers divided tasks during 1990, putting the features that could be programmed most quickly onto a fast track that would allow them to be released as Version 5.0.3 later that year.

The other half of the team worked on the rest of the planned enhancements for what would become System 6.0. When 5.0.4 was completed, the entire team again came together to continue work on this upgrade. After fourteen months of hard work, they were finally ready to release GS/OS System 6.0 in March 1992. In addition to FSTs for the Mac HFS disks, DOS 3.3, and Apple Pascal, device drivers were created to allow support of the Apple Scanner, the slot-based Apple II Memory Expansion card (which on the IIGS works primarily as a RAM disk), and the Apple Tape Drive. The SCSI drivers were enhanced, and the Apple 5.25 disk driver was made faster. A new printer driver was included, to support the Apple StyleWriter inkjet printer, and more large fonts were included to use with that and other printers. The Finder was re-designed almost from scratch by Andy Nicholas, the author of ShrinkIt and GS-ShrinkIt. Archiver (a disk backup utility) and Teach (a GS/OS-based text-editing program) were also included. Finally, ProDOS 8 v2.0.1 was released, allowing 8-bit programs access to as many as fourteen disk devices on a single slot. This made large, partitioned hard disks usable even to Apple IIc and enhanced IIe users (this version of ProDOS 8 required the opcodes of the 65c02 chip, although ProDOS 8 v1.9 was still available to run on the Apple II Plus or unenhanced IIe).<12>

At the 1992 KansasFest, Apple engineers announced that v6.0.1 of GS/OS would be out later in 1992 or early in 1993. Along with specific support of the Apple II Ethernet card, this version is expected to include bug fixes found in 6.0, and an MS-DOS FST (at least read-only, with write capability to come later).

NEXT INSTALLMENT: Languages