Macintosh users are known for a fierce antipathy toward command lines, so it seems a little surprising that they would use Perl, the archetypal command line tool. Yet MacPerl has attracted thousands of users in the four years of its existence. Macintosh users, especially programmers, are faced with similar data processing tasks as users of other platforms, so they end up picking the same tool to solve them, especially if they already know it from other platforms. A further attraction of MacPerl is that many interesting Perl applications, from James Tisdall's DNA Workbench to Thomas Boutell's PerlMUD, run on MacPerl with only a few changes. With the rise of the World Wide Web, MacPerl has also gained numerous users who use their Macs to run CGI scripts - sometimes in a production WWW environment, but often as a convenient prototyping platform.
In this article, I'm presenting the MacOS port (Perl also runs under the A/UX and MachTen Macintosh UNIX environments. However, these are conventional UNIX ports and will not be further discussed here.) of Perl 4 and 5 with a focus on implementation challenges that are unlikely to arise on UNIX platforms:
- Augmenting the standard libraries supplied by Macintosh compiler vendors with POSIX and other common UNIX functions used in the baseline Perl source.
- Making a command line based programming language attractive for using in a windowing environment.
First, though, a look back in history...
Perl on the Mac: A timeline
In Summer 1991, I was called by the Swiss army to do my compulsory 17 weeks of military basic training. While my duties were not unduly harsh, I soon experienced withdrawal symptoms from my programming habits and therefore started looking for a programming project to do on weekends. Since I had ported gawk to the Macintosh the preceding winter, and recently had become interested in Perl, I decided to attempt a port of Perl.
The port proceeded quickly, and in August 1991, I was able to report the first partial successes. Tim Endres saw one of my Usenet postings and started getting involved in cleaning up my initial attempts. When my military service ended in November 1991, I had a little more spare time for the port and in January 1992, we announced the first release of MacPerl, 4.0.2, to the public. The favorable reception of our efforts encouraged me to do further work on library functions, and in December 1992, I released version 4.0.5 which featured socket and DBM support. By that time, it had become clear that I wouldn't get rid of MacPerl-related e-mail anytime soon, so in January 1993, I created a mailing list to support it.
I kept fixing bugs in MacPerl and adding major new features in roughly annual increments since then. In October 1993, I released version 4.1.0, the first version with a true Macintosh human interface. In December 1994, version 4.1.4 started supporting Apple's PowerPC architecture, and in August 1995, version 5.0.0 finally added Perl 5 support on the Mac (Listing 1 — Running MacPerl).
Augmenting the standard libraries
One reason for the popularity Perl enjoys on UNIX platforms is the fact that it provides convenient access to all system calls available in the OS. Programmers rely extensively on these calls and expect a substantial subset of the POSIX.1 calls to be present, even on non-UNIX platforms.
The Macintosh Programmer's Workshop (MPW) C compiler, however, which was used to compile all versions of MacPerl 4, offers essentially only the C library calls mandated by the ISO C standard. Low level I/O calls (open(), close(), read(), write(), lseek(), fcntl(), ioctl()) are supported to a certain extent, but not always with POSIX.1 semantics. In particular, lseek() did not permit seeking past the end of file, which led to problems porting some DBM packages. Many other calls, even such common ones as chdir(), stat(), or opendir(), were not supported.
Thus, the porters had to write their own library functions. In the first releases of MacPerl, a few of the most severely missed functions (the opendir() and stat() families of calls) were added. Soon, however, I decided on a more ambitious project: During 1992, I developed GUSI, the Grand Unified Socket Interface (in the author's rural dialect of Swiss German, "gusi" is the word for "piglet"), a library offering the familiar BSD sockets interface on top of a number of Macintosh communications protocols, all of which had rather different application programmer interfaces:
- MacTCP, the Apple TCP/IP implementation.
- ADSP, the AppleTalk Datastream Protocol.
- PAP, the (Postscript) Printer Access Protocol.
- The System 7 Process-to-Process Communications Toolbox.
In addition to this communication functionality, I also added many POSIX.1 functions. In particular, I mapped the Macintosh concept of "Finder Alias" files to the POSIX.1 concept of symbolic links. This also necessitated modifying all library functions taking file paths to transparently resolve embedded alias files (e.g., paths of the form a:b:c, where a:b is an alias pointing to a folder), a feature not offered by most vendor-supplied libraries in 1992.
However, some areas of UNIX functionality were still not covered. The fork() and exec() functions in all forms remain unimplemented for reasons that not all MacPerl users found as obvious as I did. The memory and process management of the MacOS, while not particularly hostile to multitasking, is rather unsuited to fork()/exec() style multitasking.( These problems don't seem to be unique to the Macintosh. Almost all non-UNIX platforms seem to place considerable restrictions on the use of fork()/exec().) However, Marc Parmet's Macintosh port of Emacs 18 (ftp://ftp.cs.cornell.edu/pub/parmet/) and Jonathan Kimmit's PATMOS environment demonstrate that the problem is surmountable.
Typical Perl code also makes common use of idioms whose implementation falls into the domain of POSIX.2, such as the following:
1. open(LASER, "| lpr") 2. chop($cwd = 'pwd') 3. while (<*.c>) { ... 4. system("echo 1")
Each of these idioms presents different implementation challenges. Idiom #1 was implemented in Winter 1991/1992 by delegating the call to ToolServer, a scriptable version of Apple's Macintosh Programmer's Workshop (MPW) command line environment. ToolServer wants its input from disk files, not from pipes, but buffering the pipe (input for "| xyz", output for "xyz |") in a file proved to be quite satisfactory.
This approach worked well for backquotes in principle, but not for idiom #2 in practice, as Toolserver is not available to most MacPerl users and pwd is not by default available in Toolserver. The solution I ultimately adopted is to have the my_popen() routine check for a number of UNIX commands often (ab)used in this fashion in UNIX perl code:
- 'pwd' was handled by calling getcwd().
- 'hostname' was handled by calling gethostname().
Idiom #3, which is expanded internally by Perl to a pipe from "echo *.c", presents a similar problem: Toolserver is not widely available and it uses a different globbing syntax. I left this problem unsolved in MacPerl 4, but MacPerl 5 uses built-in globbing code to handle this case.
Idiom #4 seems similar to idiom #1, but a faithful implementation would have to redirect both the standard input and the standard output of the called code, which is not possible with the technique used for idiom #1. After a few years of deliberation, I decided to implement a pragmatic solution by ignoring standard input. Standard input to system() seems to be used rarely in practice; I don't think that standard input is used in any system() call in the Perl test suite.
A link to a display window looks like this. Keeping Your Code Portable To The MAC.
The Integrated Environment
The original Macintosh port of Perl was written for MPW, a command line oriented programming environment. MPW provides a natural habitat for tools like Perl, and I had assumed that my target audience would have MPW anyway. User feedback, however, quickly proved that the primitive standalone application I had included in the first release as a last minute thought was more popular than the MPW based version. In summer 1993, I was finally ready to address the requests for an implementation of Perl in a real Macintosh application with better online help. By fall 1993, the MacPerl user interface had a look that it kept (with minor adjustments) until today.
The figure below shows the look of MacPerl during a script debugging. The "MacPerl" window is used for the standard input/output streams, while there is a separate "Perl Debug" window for debugger interaction. Scripts can open any number of additional windows, but this feature doesn't seem to be used much in practice.
The integrated environment tries to compensate for the Mac's lack of a command line by offering a multitude of methods to run MacPerl scripts:
- Scripts without input files are started by selecting a menu command "Run Script..." and selecting the script to be executed from a standard file selection dialog.
- If a script is open in a MacPerl window, it can be executed by selecting the "Run Front Window" menu command.
- Scripts that take input files can be saved as "Droplets": Small applications that are started from the Finder by selecting the input files and dropping them on the script's icon.
- For specialized tasks (e.g., CGI scripts), MacPerl offers various "glue packages" that let Perl scripts interact with other Macintosh applications. The MacPerl application also allowed me to experiment with various online help mechanisms. Online help in MacPerl4 is based on a modified ASCII version of the standard Perl manual page. Selecting a built-in function and hitting the "Help" key dumps the appropriate manual section into a text window. This system works, but the amount of post-processing required makes keeping up with changes in the standard distribution of the manual too much work.
That's why I adopted a different approach for the MacPerl5 help system: Help requests are translated into relative URLs in the HTML version of the Perl 5 manual, which are then, by means of the Internet Config system,(Internet Config, written by Peter Lewis and Quinn, is a freeware application offering a central database of an user's Internet settings (his e-mail address, his home page, his favorite WWW viewer).) sent to the users preferred WWW browser. In addition to the help index, which contains all built-in functions and variables, the MacPerl help menu (shown on this page) also offers direct access to all chapters of the Perl 5 manual and to the MacPerl WWW home page. To avoid network traffic, the generated URLs are in general file:/ URLs pointing to a local disk copy of the manual.
Future Plans
Although MacPerl is currently still based on perl5.001m, the core code seems to have stabilized somewhat. The major challenges now seem to be in the embedding of Perl and in work on the Perl library.
The integrated environment described in the previous section has well known weaknesses; in particular, the built in editor is limited to 32K characters and is rather inadequate for serious development work. I feel, though, that attempts to compete against powerful text editors like Alpha and BBEdit would be pointless. Instead, I'd like to improve the integration of MacPerl into these editors and into any other context desired by Mac users. In the short term, MacPerl has worked toward that goal through glue code for BBEdit and Alpha (aided by a MacPerl support package for Alpha written in Tcl by Tom Pollard). Glue code is also available for the various Macintosh HTTP servers (MacHTTP, WebSTAR, and Netpresenz) to provide a CGI compatible environment for Perl scripts.
With System 7, Apple introduced the Open Scripting Architecture (OSA), a common API for automating tasks in Macintosh applications. MacPerl currently takes advantage of the OSA by supporting the embedding of AppleScript fragments in Perl scripts. In the long term, however, I would like MacPerl itself to become an OSA language and provide a third alternative to AppleScript and UserLand Software's Frontier language. Another important emerging Macintosh technology is the OpenDoc component architecture, where MacPerl has considerable potential both as a scripting language and as a language for the creation of new OpenDoc components. (OpenDoc is based on IBM's System Object Model (SOM) technology. Unfortunately, although SOM is designed to be language independent, there is currently no Macintosh support for languages other than C and C++.)
The success of Perl 5 and of the module coordination efforts like CPAN and Tim Bunce's Module List has become evident in the vast number of new useful library modules being created. This is a somewhat mixed blessing to me as a porter, as I never even got around to port all of the comparatively tiny bundled Perl 4 library. Luckily, though, it looks as if others will start cleaning up the "99% MacPerl compatible" libraries and get them to run.
The extension mechanisms of Perl 5 also represent a potentially huge opportunity to provide full access to the MacOS API. Currently, there are probably more than 2,500 API calls; writing good Perl interfaces to all of them will require a nontrivial amount of work, but I'm optimistic that here, too, some other MacPerl users will join in the effort. At the moment, there are preliminary interfaces to the Memory manager, the Resource manager, and the AppleEvent manager. The DynaLoader module is already implemented on PowerPC Macs and will be implemented on older Macs soon, which makes developing and testing MacPerl extensions a lot easier.
As on many other platforms, Perl has found good use and enthusiastic users on the Apple Macintosh, and I'm optimistic that it will continue to expand its usefulness and user base. As on other platforms, the success of MacPerl and the life of its primary porter call for the work to be distributed on more shoulders, but I'm optimistic that this, too, will happen.
I am indebted to Janis Greenberg, Sandra Silcot, and Hal Wine for their valuable comments on this article.
Matthias Neeracher is a Perl hacker trapped in a graduate student's body. He never leaves home without a Swiss Army Knife in his pocket.