                           Binary ][ protocol
 
                              developed by
                             Gary B. Little
 
 Version History
 ---------------
 November 24, 1986  : Initial release.
 
 Background
 ----------
 Transferring Apple II files in binary form to commercial information
 services like CompuServe, Delphi, GEnie, and The Source is, to put it
 mildly, a frustrating exercise. (For convenience, I'll refer to such
 services, and any other non-Apple II systems, as "hosts.") Although
 most hosts are able to receive a file's *data* in binary form (using
 the Xmodem protocol, for example), they don't receive the file's all-
 important attribute bytes. All the common Apple II operating systems,
 notably ProDOS, store the attributes inside the disk directory, not
 inside the file itself.
 
 The ProDOS attributes are the access code, file type code, auxiliary
 type code, storage type code, date of creation and last modification,
 time of creation and last modification, the file size, and the name of
 the file itself. (All these terms are defined in Apple's "ProDOS
 Technical Reference Manual" or in the book "Apple ProDOS: Advanced
 Features for Programmers" by Gary Little.) It is usually not possible
 to use a ProDOS file's data without knowing what the file's attributes
 are (particularly the file type code, auxiliary type code, and size).
 This means ProDOS files uploaded in binary form to a host are useless
 to those who download them. The same is true for DOS 3.3 and Pascal
 files.
 
 Most Apple II communications programs use special protocols for
 transferring file attributes during a binary file transfer, but none
 of these protocols have been implemented by hosts. These programs are
 only useful for exchanging files with another Apple II running the
 same program.
 
 At present, the only acceptable way to transfer an Apple II file to a
 host is to convert it into lines of text and send it as a textfile.
 Such a textfile would contain a listing of an Applesoft program, or a
 series of Apple II system monitor "enter" commands (e.g., 0300:A4 32
 etc.). Someone downloading such a file can convert it to binary form
 using the Applesoft EXEC command.
 
 The main disadvantage of this technique is that the text version of
 the file is over three times the size of the original binary file,
 making it expensive (in terms of time and $$$) to upload and download.
 It is also awkward, and sometimes impossible, to perform the binary-
 to-text or text-to-binary conversion.
 
 The solution to the problem is to upload an encoded binary file which
 contains not just the file's data, but the file's attributes as well.
 Someone downloading such a file, say using Xmodem, can then use a
 conversion program to strip the attributes from the file and create a
 file with the required attributes.
 
 To make this technique truly useful, however, the Apple II community
 must agree on a format for this encoded binary file. A variety of
 incompatible formats, all achieving the same general result, cannot be
 allowed to appear.
 
 It is proposed that the Binary II format described in this document be
 adopted. What follows is a description of the Binary II format in
 sufficient detail to allow software developers to implement it in
 Apple II communications programs.
 
 The Binary II File Format
 -------------------------
 The Binary II form of a standard file consists of a 128-byte file
 information header followed by the file's data. The data portion of
 the file is padded with nulls ($00 bytes), if necessary, to ensure the
 data length is an even multiple of 128. As a result, the Binary II
 form of a file is never more than 255 bytes longer than the original
 file.
 
 The file information header contains four ID bytes, the attributes of
 the file (in ProDOS 8 form), and some control information. Here is the
 structure of the header:
 
       Offset  Length                  Contents
       ------  ------   ---------------------------------------
        +0       1      ID byte: always $0A
        +1       1      ID byte: always $47
        +2       1      ID byte: always $4C
        +3       1      access code
        +4       1      file type code
        +5       2      auxiliary type code
        +7       1      storage type code
        +8       2      size of file in 512-byte blocks
        +10      2      date of modification
        +12      2      time of modification
        +14      2      date of creation
        +16      2      time of creation
        +18      1      ID byte: always $02
        +19      1      [reserved]
        +20      3      end-of-file (EOF) position
        +23      1      length of filename/partial pathname
        +24      64     ASCII filename or partial pathname
        +88      23     [reserved, must be zero]
        +111     1      ProDOS 16 access code (high)
        +112     1      ProDOS 16 file type code (high)
        +113     1      ProDOS 16 storage type code (high)
        +114     2      ProDOS 16 size of file in blocks (high)
        +116     1      ProDOS 16 end-of-file position (high)
        +117     4      disk space needed
        +121     1      operating system type
        +122     2      native file type code
        +124     1      phantom file flag
        +125     1      data flags
        +126     1      Binary II version number
        +127     1      number of files to follow
 
 Multi-byte numeric quantities are stored with their low-order bytes
 first, the same order expected by ProDOS. All reserved bytes must be
 set to zero; they may be used in future versions of the protocol.
 
 To determine the values of the attributes to be put into a file
 information header for a ProDOS file, you can use the ProDOS
 GET_FILE_INFO and GET_EOF MLI commands.
 
    Note: Some file attributes returned by ProDOS 16 commands
          are one or two bytes longer than the attributes
          returned by the corresponding ProDOS 8 commands. At
          present, these extra bytes are always zero, and
          probably will remain zero forever. In any event,
          place the extra bytes returned by ProDOS 16 in the
          header at +114 to +119. ProDOS 8 communications
          programs should zero these header locations.
 
 The "disk space needed" bytes contain the number of 512-byte disk
 blocks the files inside the Binary II file will occupy after they've
 been removed from the Binary II file. (The format of a Binary II file
 containing multiple files is described below.) If the number is zero,
 the person uploading the file did not bother to calculate the space
 needed. The "disk space needed" must be placed in the file information
 header for the first file inside the Binary II file; it can be set to
 zero in subsequent headers. A downloading program can inspect "disk
 space needed" and abort the transfer immediately if there isn't enough
 disk free space.
 
 The value of the "operating system type" byte indicates the native
 operating system of the file:
 
         $00 = ProDOS 8, ProDOS 16, or SOS
         $01 = DOS 3.3
         $02 = Pascal
         $03 = CP/M
         $04 = MS-DOS
 
 Note that even if a file is not a ProDOS file, the attributes in the
 file information header, including the name, must be inserted in
 ProDOS form. Instructions on how to do this for DOS 3.3 files are
 given later in this document. Similar considerations apply for the
 files of other operating systems.
 
 The "native file type code" has meaning only if the "operating system
 type" is non-zero. It is set to the actual file type code assigned to
 the file by its native operating system. (Some operating systems, such
 as CP/M and MS-DOS, do not use file type codes, however.) Contrast
 this with the file type code at +4, which is the closest equivalent
 ProDOS file type code. The "native file type code" is needed to
 distinguish files which have the same *ProDOS* file type, but which
 may have different file types in their native operating system. Note
 that if the file type code is only byte long (the usual case), the
 high-order byte of "native file type code" is set to zero.
 
 The "phantom file flag" byte indicates whether a receiver of the
 Binary II file should save the file which follows (flag is zero) or
 ignore it (flag is non-zero). It is anticipated that some
 communications programs will use phantom files to pass non-essential
 explanatory notes or encoded information which would be understood
 only by a receiver using the same communications program. Such
 programs must not rely on receiving a phantom file, however, since
 this would mean they couldn't handle Binary II files created by other
 communications programs.
 
 The first two bytes in a phantom file *must* contain an ID code unique
 to the communications program. Developers must obtain ID codes from
 Gary Little to ensure uniqueness (see below for his address). Here is
 a current list of approved ID codes for phantom files used by Apple II
 communications programs:
 
         $00 $00  =  [generic]
         $00 $01  =  Point-to-Point
         $00 $02  =  Tele-Master Communications System
 
 Developers of communications programs are responsible for defining and
 publishing the structures of their phantom files.
 
 The ID bytes appear in the first two bytes of the phantom file.
 Phantom files having a generic ID code of zero must contain lines of
 text terminated by a $00 byte. The text must begin at the third byte
 in the file.
 
 The "data flags" byte is a bit vector indicating whether the data
 portion of the Binary II file has been compressed, encrypted, or
 packed. If bit 7 (the high-order bit) is set to 1, the file is
 compressed. If bit 6 is 1, the file is encrypted. If bit 0 is 1, the
 file is a sparse file that is packed. A Binary II downloading program
 can examine this byte and warn the user, when necessary, that the file
 must be expanded, decrypted, or unpacked. The person uploading a
 Binary II file may use any convenient method for compressing,
 encrypting, or packing the file but is responsible for providing
 instructions on how to restore the file to its original state.
 
 This initial release of Binary II has a "Binary II version number" of
 $00.
 
 Handling Multiple Files
 -----------------------
 An appealing feature of Binary II is that a single Binary II file can
 hold multiple disk files, making it easy to keep a group of related
 files "glued" together when they're sent to a host.
 
 The structure of a Binary II file containing multiple disk files is
 what you might expect: it is a series of images of individual Binary
 II files. For example, here is the general structure of a Binary II
 file containing three disk files:
 
  start                                                           end
  -------------------------------------------------------------------
  | Header #1 | #1 Data | Header #2 | #2 Data | Header #3 | #3 Data |
  -------------------------------------------------------------------
    +127 = 2              +127 = 1              +127 = 0
 
 The data areas following each header end on a 128-byte boundary.
 
 The "number of files to follow" byte (at offset 127) in the file
 information header for each disk file contains the number of disk
 files that follow it in the Binary II file. It will be zero in the
 header for the last disk file in the group.
 
 Filenames and Partial Pathnames
 -------------------------------
 Notice that you can put a standard ProDOS filename or a partial
 pathname in the file information header (but never a complete
 pathname). *Beware!* Don't use a partial pathname unless you've
 included, earlier on in the Binary II file, file information headers
 for each of the directories referred to in the partial pathname. Such
 a header must have its "end of file position" bytes set to zero, and
 no data blocks for the subdirectory file must follow it.
 
 For example, if you want to send a file whose partial pathname is
 HELP/GS/READ.ME, first send a file information header defining the
 HELP/ subdirectory, then one defining the HELP/GS/ subdirectory. If
 you don't, someone downloading the Binary II file won't be able to
 convert it because the necessary subdirectories will not exist.
 
 Filename Convention
 -------------------
 Whenever a file is sent to a host, the host asks the sender to provide
 a name for it. If it's a Binary II file, the name provided should end
 in .BNY so that its special form will be apparent to anyone viewing a
 list of filenames.
 
 Identifying Binary II Files
 ---------------------------
 ose the ProDOS
 file. You would repeat this for each file contained inside the Binary
 II file.
 
    Note: The number of 128-byte data blocks following the
          file information header must be derived from the
          "end-of-file position" attribute (EOF) not the "size
          of file in blocks" attribute. Calculate the number
          by dividing EOF by 128 and adding one to the result
          if EOF is not 0 or an exact multiple of 128.
 
 Exception: If the file information header defines a subdirectory (the
 file type code is 15), simply CREATE the subdirectory file. Do not
 OPEN it and do not set its size with SET_EOF.
 
 Ideally, all this conversion work will be done automatically by a
 communications program during an Xmodem (or other binary protocol)
 download. If not, a separate conversion program will have to be run
 after the Binary II file has been received and saved to disk. Gary
 Little has published a public domain program, called BINARY.DWN, that
 will do this for you. (A related program, BINARY.UP, combines multiple
 ProDOS files into one Binary II file which can then be uploaded to a
 host.)
 
 DOS 3.3 Considerations
 ----------------------
 With a little extra effort, you can also convert DOS 3.3 files to
 Binary II form. This involves translating the DOS 3.3 file attributes
 to the corresponding ProDOS attributes so that you can build a proper
 file information header. Here is how to do this:
 
    (1) Set the name to one that adheres to the stricter ProDOS naming
        rules.
 
    (2) Set the ProDOS file type code, auxiliary type code, and access
        code to values which correspond to the DOS 3.3 file type:
 
           DOS 3.3  |   ProDOS     ProDOS    ProDOS
          file type | file type   aux type   access
         -----------|----------- ---------- --------
          $00 ( T)  | $04 (TXT)    $0000      $E3
          $80 (*T)  | $04 (TXT)    $0000      $21
          $01 ( I)  | $FA (INT)    $0C00      $E3
          $81 (*I)  | $FA (INT)    $0C00      $21
          $02 ( A)  | $FC (BAS)    $0801      $E3
          $82 (*A)  | $FC (BAS)    $0801      $21
          $04 ( B)  | $06 (BIN)     (*)       $E3
          $84 (*B)  | $06 (BIN)     (*)       $21
          $08 ( S)  | $06 (BIN)    $0000      $E3
          $88 (*S)  | $06 (BIN)    $0000      $21
          $10 ( R)  | $FE (REL)    $0000      $E3
          $90 (*R)  | $FE (REL)    $0000      $21
          $20 ( A)  | $06 (BIN)    $0000      $E3
          $A0 (*A)  | $06 (BIN)    $0000      $21
          $40 ( B)  | $06 (BIN)    $0000      $E3
          $C0 (*B)  | $06 (BIN)    $0000      $21
 
          (*) Set the aux type for a B file to the
              value stored in the first two bytes
              of the file (this is the default load
              address).
 
     (3) Set the storage type code to $01.
 
     (4) Set the size of file in blocks, date of creation, date of
         modification, time of creation, and time of modification to
         $0000.
 
     (5) Set the end-of-file position to the length of the DOS 3.3
         file, in bytes. For a B file (code $04 or $84), this number is
         stored in the third and fourth bytes of the file. For an I
         file (code $01 or $81) or an A file (code $02 or $82), this
         number is stored in the first and second bytes of the file.
 
     (6) Set the operating system type to $01.
 
     (7) Set the native file type code to the value of the DOS 3.3 file
         type code.
 
 Attribute bytes inside a DOS 3.3 file (if any) must *not* be included
 in the data portion of the Binary II file. This includes the first
 four bytes of a B (Binary) file, and the first two bytes of an A
 (Applesoft) or I (Integer BASIC) file.
 
 Acknowledgements
 ----------------
 Thanks to Glen Bredon for suggesting that partial pathnames be allowed
 in file information headers. Thanks also to Shawn Quick for suggesting
 the "phantom file" byte, to Scott McMahan for suggesting the
 compression and encryption bits in the "data flags" byte, and to
 William Bond for suggesting the "disk space needed" bytes. Finally, a
 big thank you to Neil Shapiro, Chief Sysop of MAUG, for supporting the
 development of the Binary II format and helping it become a true
 standard.
 
 Feedback and Support
 --------------------
 Send any comments or questions concerning the Binary II file format
 to:
 
    Gary B. Little
    #210 - 131 Water Street
    Vancouver, British Columbia
    Canada  V6B 4M3
    (604) 681-3371
 
    CompuServe : 70135,1007
    Delphi     : GBL
    MCI Mail   : 658L6
 
 Gary developed the Point-to-Point telecommunications program published
 by Pinpoint Publishing. He has also written several books on how to
 program Apple computers: "Inside the Apple IIe," "Inside the Apple
 IIc," "Apple ProDOS: Advanced Features for Programmers," and "Mac
 Assembly Language: A Guide for Programmers." He is currently a
 Contributing Editor for A+ magazine and writes A+'s monthly Rescue
 Squad column. Gary has also published articles in Nibble, Micro, Call
 -A.P.P.L.E, and Softalk.
 
