Preface
`IO`
The purpose of this package is to allow efficient and flexible input/output operations from &GAP;. This is achieved by providing bindings to the low-level I/O functions in the C-library. On top of this an implementation of buffered I/O in the &GAP; language is provided. Further, a framework for serialisation of arbitrary &GAP; objects is implemented. Finally, an implementation of the client side of the HTTP protocol is included in the package.

This package allows to use file based I/O, access to links and file systems, pipes, sockets, and the UDP and TCP/IP protocols.

By default the `IO` package is not automatically loaded by &GAP; when it is installed. You must load the package with LoadPackage("IO"); before its functions become available.

Please, send me an e-mail (neunhoef@mcs.st-and.ac.uk) if you have any questions, remarks, suggestions, etc. concerning this package. Also, I would like to hear about applications of this package.

Max Neunhöffer
Installation of the `IO`-package To get the newest version of this &GAP; 4 package download one of the archive files io-x.x.tar.gz io-x.x.tar.bz2 io-x.x.zip and unpack it using gunzip io-x.x.tar.gz; tar xvf io-x.x.tar or bzip2 -d io-x.x.tar.bz2; tar xvf io-x.x.tar or unzip -x io-x.x.zip respectively.

Do this in a directory called pkg, preferably (but not necessarily) in the pkg subdirectory of your &GAP; 4 installation. It creates a subdirectory called io.

The package will not work without the following compilation step.

To compile the C part of the package do (in the pkg directory) cd io ./configure make If you installed the package in another pkg directory than the standard pkg directory in your &GAP; 4 installation, then you have to do two things. Firstly during compilation you have to use the option --with-gaproot=PATH of the configure script where PATH is a path to the main &GAP; root directory (if not given the default ../.. is assumed).

Secondly you have to specify the path to the directory containing your pkg directory to &GAP;'s list of directories. This can be done by starting &GAP; with the -l command line option followed by the name of the directory and a semicolon. Then your directory is prepended to the list of directories searched. Otherwise the package is not found by &GAP;. Of course, you can add this option to your &GAP; startup script.

If you installed &GAP; on several architectures, you must execute the configure/make step for each of the architectures. You can either do this immediately after configuring and compiling &GAP; itself on this architecture, or alternatively (when using version 4.5 of &GAP; or newer) set the environment variable CONFIGNAME to the name of the configuration you used when compiling &GAP; before running ./configure. Note however that your compiler choice and flags (environment variables CC and CFLAGS) need to be chosen to match the setup of the original &GAP; compilation. For example you have to specify 32-bit or 64-bit mode correctly!

Static linking This feature does not work in this version of the package. We leave the old documentation here for the case that the feature will be reenabled inthe future.

This might be interesting for M$ Windows users, as dynamic loading of binary modules does not always work there. You can also create a new statically linked gap binary as follows:

Go into the main &GAP; directory and then into bin/BINDIR. Here BINDIR means the directory containing the gap executable after compiling gap. This directory also contains the &GAP; compiler script gac. Assuming IO in the standard location you can then say ./gac -o gap-static -p "-DIOSTATIC -I../../pkg/io/bin/BINDIR" \ -P "-static" ../../pkg/io/src/io.c Then copy your gap start script to, say, gaps and change the references to the &GAP; binary to gap-static.

Note that you have to replace BINDIR by the name containing the gap executable after compiling GAP as above. If you have installed the package in a different place than the standard, you have to replace ../.. in the above command by the path to the directory containing the pkg directory into which you installed `IO`. If you want to install more than one package with a C-part like this package, you can still create a statically linked &GAP; executable by combining all the compile and link options and all the .c files as in the ./gac command above. For the IO package, you have to add -DIOSTATIC -I../../pkg/io/bin/BINDIR to the string of the -p option and the file ../../pkg/io/src/io.c somewhere on the command line. As above, ../.. and BINDIR have to be replaced if you installed in a non-standard location.
Recompiling the documentation Recompiling the documentation is possible by the command gap makedoc.g in the io directory. But this should not be necessary.
Functions directly available from the C library The following functions from the C library are made available as &GAP; functions:

accept, bind, chdir, chmod, chown, close, closedir, connect, creat, dup, dup2, execv, execve, execvp, exit, fchmod, fchown, fcntl, fork, fstat, gethostbyname, gethostname, getpid, getppid, getsockname, getsockopt, gettimeofday, gmtime, kill, lchown, link, listen, localtime, lseek, lstat, mkdir, mkfifo, mknod, open, opendir, pipe, read, readdir, readlink, recv, recvfrom, rename, rewinddir, rmdir, seekdir, select, send, sendto, setsockopt, socket, stat, symlink, telldir, unlink, write.

Use the man command in your shell to get information about these functions.

For each of these functions there is a corresponding &GAP; global function with the prefix IO&uscore; before its name. Apart from minor differences (see below) they take exactly the same arguments as their C counterparts. Strings must be specified as &GAP; strings and integers as &GAP; immediate integers. Return values are in general the same as for the C counterparts. However, an error condition is indicated by the value fail instead of -1, and if the result can only be success or failure, true indicates success.

All errors are reported via the function.

In the C library a lot of integers are defined as macros in header files. All the necessary values for the above functions are bound to their name in the global IO record.

Warning: Existence of many of these functions and constants is platform dependent. The compilation process checks existence and this leads to the situation that on the &GAP; levels the functions and constants are there or not. If you want to develop platform independent &GAP; code using this package, then you have to check for existence of the functions and constants you need.
Differences in arguments - an overview The open function has to be called with three arguments. The version with two arguments is not available on the &GAP; level.

The read function takes four arguments: fd is an integer file descriptor, st is a &GAP; string, offset is an offset within this string (zero based), and count is the maximal number of bytes to read. The data is read and stored into the string st, starting at position offset+1. The string st is made long enough, such that count bytes would fit into it, beginning at position offset+1. The number of bytes read is returned or fail in case of an error.

The write function is similar, it also takes four arguments: fd is an integer file descriptor, st is a &GAP; string, offset is an offset within this string (zero based), and count is the number of bytes to write, starting from position offset+1 in the string st. The number of bytes written is returned, or a fail in case of an error.

The opendir function only returns true or fail.

The readdir function takes no argument. It reads the directory that was specified in the last call to opendir. It just returns a string, which is the name of a file or subdirectory in the corresponding directory. It returns false after the last file name in the directory or fail in case of an error.

The closedir function takes no argument. It should be called after readdir returned false or fail to avoid excessive use of file descriptors.

The functions stat, fstat, and lstat only take one argument and return a &GAP; record that has the same entries as a struct stat.

The function socket can optionally take a string as third argument. In that case it automatically calls getprotobyname to look up the protocol name.

The functions bind and connect take only one string argument as address field, because the string already encodes the length.

There are two convenience functions and to create such addresses. The first takes two arguments addr and port, where addr is a string of length 4, containing the 4 bytes of the IP address and port is a port number as &GAP; integer. The function takes the same arguments, but the first can be a string containing an IP address in dot notation like 137.226.152.77 or a hostname to be looked up.

The setsockopt function has no argument optlen. The length of the string optval is taken.

The select function works as the function UNIXSelect in the &GAP; library.

As of now, the file locking mechanisms of fcntl using struct flock are not yet implemented on the &GAP; level.
The low-level functions in detail Nearly all of this functions return an integer result in the C library. On the &GAP; level this is either returned as a non-negative integer in case of success or as fail in case of an error (where on the C level -1 would be returned). If the integer can only be 0 for no error this is changed to true on the &GAP; level. an integer or fail Accepts an incoming network connection. For details see man 2 accept. The argument addr can be made with and contains its length such that no third argument is necessary. an integer or fail Binds a local address to a socket. For details see man 2 bind. The argument my&uscore;addr can be made with and contains its length such that no third argument is necessary. true or fail Changes the current working directory. For details see man 2 chdir. true or fail Changes the mode of a file. For details see man 2 chmod. true or fail Sets owner and/or group of file. For details see man 2 chown. true or fail Closes a file descriptor. For details see man 2 close. true or fail Closes a directory. For details see man 3 closedir. Has no arguments, because we only have one DIR struct in the C part. true or fail Connects to a remote socket. For details see man 2 connect. The argument serv&uscore;addr can be made with and contains its length such that no third argument is necessary. an integer or fail Creates a new file. For details see man 2 creat. an integer or fail Duplicates a file descriptor. For details see man 2 dup. true or fail Duplicates a file descriptor to a new one. For details see man 2 dup2. fail or does not return Replaces the process with another process. For details see man 3 execv. The argument argv is a list of strings. The called program does not have to be the first argument in this list. fail or does not return Replaces the process with another process. For details see man 3 execve. The arguments argv and envp are both lists of strings. The called program does not have to be the first argument in argv. The list envp can be made with from a record acquired from and modified later. fail or does not return Replaces the process with another process. For details see man 3 execvp. The argument argv is a list of strings. The called program does not have to be the first argument in this list. Stops process immediately with return code status. For details see man 2 exit. The argument status must be an integer. Does not return. true or fail Changes mode of an opened file. For details see man 2 fchmod. true or fail Changes owner and/or group of an opened file. For details see man 2 fchown. an integer or fail Does various things to control the behaviour of a file descriptor. For details see man 2 fcntl. an integer or fail Forks off a child process, which is an identical copy. For details see man 2 fork. Note that if you want to use the function to wait or check for the termination of child processes, you have to activate the SIGCHLD handler for this package beforehand by using the function . Note further that after that you cannot use the function any more, since its SIGCHLD handler does not work any more. To switch back to that functionality use the function . a record or fail Returns the file meta data for an opened file. For details see man 2 fstat. A &GAP; record is returned with the same entries than a struct stat. a record or fail Return host information by name. For details see man 3 gethostbyname. A &GAP; record is returned with all the relevant information about the host. a string or fail Return host name. For details see man 3 gethostname. an integer Returns the process ID of the current process as an integer. For details see man 2 getpid. an integer Returns the process ID of the parent of the current process as an integer. For details see man 2 getppid. a string or fail Get a socket name. For details see man 2 getsockname. true or false Get a socket option. For details see man 2 getsockopt. Note that the argument optval carries its length around, such that no 5th argument is necessary. A record with components tv_sec and tv_usec This returns the time elapsed since 1.1.1970, 0:00 GMT. The component tv_sec contains the number of full seconds and the number tv_usec the additional microseconds. A record The argument is the number of seconds that have elapsed since 1.1.1970, 0:00 GMT. The result is a record with the current Greenwich mean time broken down into date and time as in the C-library function gmtime. true or fail Sends the signal sig to the process with process ID pid. For details see man 2 kill. The signal numbers available can be found in the global IO record with names like SIGTERM. true or false Changes owner and/or group of a file not following links. For details see man 2 lchown. true or false Create a hard link. For details see man 2 link. true or false Switch a socket to listening. For details see man 2 listen. A record The argument is the number of seconds that have elapsed since 1.1.1970, 0:00 GMT. The result is a record with the current local time broken down into date and time as in the C-library function localtime. an integer or fail Seeks within an open file. For details see man 2 lseek. a record or fail Returns the file meta data for a file not following links. For details see man 2 lstat. A &GAP; record is returned with the same entries than a struct stat. true or false Creates a directory. For details see man 2 mkdir. true or false Creates a FIFO special file (a named pipe). For details see man 3 mkfifo. true or false Create a special or ordinary file. For details see man 2 mknod. an integer or fail Open and possibly create a file or device. For details see man 2 open. Only the variant with 3 arguments can be used. true or false Opens a directory. For details see man 3 opendir. Note that only true is returned if everything is OK, since only one DIR struct is stored on the C level and thus only one directory can be open at any time. a record or fail Create a pair of file descriptors with a pipe between them. For details see man 2 pipe. Note that no arguments are needed. The result is either fail in case of an error or a record with two components toread and towrite bound to the two filedescriptors for reading and writing respectively. an integer or fail Reads from file descriptor. For details see man 2 read. Note that there is one more argument offset to specify at which position in the string st the read data should be stored. Note that offset zero means at the beginning of the string, which is position 1 in &GAP;. The number of bytes read or fail in case of an error is returned. a string or fail or false Reads from a directory. For details see man 2 readdir. Note that no argument is required as we have only one DIR struct on the C level. If the directory is read completely false is returned, and otherwise a string. An error is indicated by fail. an integer or fail Reads the value of a symbolic link. For details see man 2 readlink. buf is modified. The new length of buf is returned or fail in case of an error. an integer or fail Receives data from a socket. For details see man 2 recv. Note the additional argument offset which plays the same role as for the function. an integer or fail Receives data from a socket with given address. For details see man 2 recvfrom. Note the additional argument offset which plays the same role as for the function. The argument addr can be made with and contains its length such that no 7th argument is necessary. true or false Renames a file or moves it. For details see man 2 rename. true or fail Rewinds a directory. For details see man 2 rewinddir. Note that no argument is required as we have only one DIR struct on the C level. Returns fail only, if no prior command has been called. true or fail Removes an empty directory. For details see man 2 rmdir. true or fail Sets the position of the next readdir call. For details see man 3 seekdir. Note that no second argument is required as we have only one DIR struct on the C level. an integer or fail Used for I/O multiplexing. For details see man 2 select. inlist, outlist and exclist are lists of filedescriptors, which are modified. If the corresponding file descriptor is not yet ready, it is replaced by fail. The timeout values timeoutsec and timeoutusec correspond to the usual arguments of select, if both are immediate integers, they are set, otherwise select is called with no timeout value. an integer or fail Sends data to a socket. For details see man 2 send. Note that the additional argument offset specifies the position of the data to send within the string st. It is zero based, meaning that zero indicates the start of the string, which is position 1 in &GAP;. an integer or fail Sends data to a socket. For details see man 2 sendto. Note that the additional argument offset specifies the position of the data to send within the string st. It is zero based, meaning that zero indicates the start of the string, which is position 1 in &GAP;. The argument addr can be made with and contains its length such that no 7th argument is necessary. true or fail Sets a socket option. For details see man 2 setsockopt. Note that the argument optval carries its length around, such that no 5th argument is necessary. an integer or fail Creates a socket, an endpoint for communication. For details see man 2 socket. There is one little special: On systems that have getprotobyname you can pass a string as third argument protocol which is automatically looked up by getprotobyname. a record or fail Returns the file metadata for the file pathname. For details see man 2 stat. A &GAP; record is returned with the same entries than a struct stat. true or fail Creates a symbolic link. For details see man 2 symlink. an integer or fail Return current location in directory. For details see man 3 telldir. Note that no second argument is required as we have only one DIR struct on the C level. true or fail Delete a name and possibly the file it refers to. For details see man 2 unlink. a record or fail Waits for the termination of a child process. For details see man 2 waitpid. Returns a &GAP; record describing PID and exit status. The second argument wait must be either true or false. In the first case, the call blocks until new information about a terminated child process is available. In the second case no such waiting is performed, the call returns immediately. See . an integer or fail Writes to a file descriptor. For details see man 2 write. Note that the additional argument offset specifies the position of the data to send within the string st. It is zero based, meaning that zero indicates the start of the string, which is position 1 in &GAP;.
Further C level functions The following functions do not correspond to functions in the C library, but are there to provide convenience to use other functions: a string or fail Makes a struct sockaddr&uscore;in from IP address and port. The IP address must be given as a string of length four, containing the four bytes of an IPv4 address in natural order. The port must be a port number. Returns a string containing the struct, which can be given to all functions above having an address argument. a list of strings For details see man environ. Returns the current environment as a list of strings of the form key=value. true or false Installs our SIGCHLD handler. This functions works as an idempotent. That is, calling it twice does exactly the same as calling it once. It returns true when it is called for the first time since then a pointer to the old signal handler is stored in a global variable. See . Restores the original SIGCHLD handler. This function works as an idempotent. That is, calling it twice does exactly the same as calling it once. It returns true when it is called for the first time after calling . See .
High level functions for buffered I/O The functions in the previous sections are intended to be a possibility for direct access to the low level I/O functions in the C library. Thus, the calling conventions are strictly as in the original.

The functionality described in this section is implemented completely in the &GAP; language and is intended to provide a good interface for programming in &GAP;. The fundamental object for I/O on the C library level is the file descriptor, which is just a non-negative integer representing an open file of the process. The basic idea is to wrap up file descriptors in &GAP; objects that do the buffering.

Note that considerable care has been taken to ensure that one can do I/O multiplexing with buffered I/O. That is, one always has the possibility to make sure before a read or write operation, that this read or write operation will not block. This is crucial when one wants to serve more than one I/O channel from the same (single-threaded) &GAP; process. This design principle sometimes made it necessary to have more than one function for a certain operation. Those functions usually differ in a subtle way with respect to their blocking behaviour.

One remark applies again to nearly all functions presented here: If an error is indicated by the returned value fail one can use the library function to find out more about the cause of the error. This fact is not mentioned with every single function.
Types and the creation of File objects The wrapped file objects are in the following category: true or false The category of File objects. To create objects in this category, one uses the following function: a File object The argument fd must be a file descriptor (i.e. an integer) or -1 (see below).

rbufsize can either be false for unbuffered reading or an integer buffer size or a string. If it is an integer, a read buffer of that size is used. If it is a string, then fd must be -1 and a File object that reads from that string is created.

wbufsize can either be false for unbuffered writing or an integer buffer size or a string. If it is an integer, a write buffer of that size is used. If it is a string, then fd must be -1 and a File object that appends to that string is created.

The result of this function is a new File object. A convenient way to do this for reading or writing of files on disk is the following function: a File object or fail The argument filename must be a string specifying the path name of the file to work on. mode must also be a string with possible values r, w, or a, meaning read access, write access (with creating and truncating), and append access respectively. If mode is omitted, it defaults to r. bufsize, if given, must be a positive integer or false, otherwise it defaults to IO.DefaultBufSize. Internally, the function is used and the result file descriptor is wrapped using with bufsize as the buffer size.

The result is either fail in case of an error or a File object in case of success. Note that there is a similar function which also creates a File object but with additional functionality with respect to a pipeline for filtering. It is described in its section in Section . There is some more low-level functionality to acquire open file descriptors. These can be wrapped into File objects using .
Reading and writing Once a File object is created, one can use the following functions on it: a string or fail This function reads all data from the file f until the end of file. The data is returned as a &GAP; string. If the file is already at end of file, an empty string is returned. If an error occurs, then fail is returned. Note that you still have to call on the File object to properly close the file later. a string or fail This function gets two arguments, the first argument f must be a File object and the second argument len must be a positive integer. The function tries to read len bytes and returns a string of that length. If and only if the end of file is reached earlier, fewer bytes are returned. If an error occurs, fail is returned. Note that this function blocks until either len bytes are read, or the end of file is reached, or an error occurs. For the case of pipes or internet connections it is possible that currently no more data is available, however, by definition the end of file is only reached after the connection has been closed by the other side! a string or fail This function gets exactly one argument, which must be a File object f. It reads one line of data, where the definition of line is operating system dependent. The line end character(s) are included in the result. The function returns a string with the line in case of success and fail in case of an error. In the latter case, one can query the error with .

Note that the reading is done via the buffer of f, such that this function will be quite fast also for large amounts of data.

If the end of file is hit without a line end, the rest of the file is returned. If the file is already at end of file before the call, then a string of length 0 is returned. Note that this is not an error but the standard end of file convention! a list of strings or fail This function gets one or two arguments, the first of which must always be a File object f. It reads lines of data (where the definition of line is operating system dependent) either until end of file (without a second argument) or up to max lines (with a second argument max. A list of strings with the result is returned, if everything went well and fail oterwise. In the latter case, one can query the error with .

Note that the reading is done via the buffer of f, such that this function will be quite fast also for large amounts of data.

If the file is already at the end of file, the function returns a list of length 0. Note that this is not an error but the standard end of file convention! true or false This function takes one argument f which must be a File object. It returns true or false according to whether there is data to read available in the file f. A return value of true guarantees that the next call to on that file will succeed without blocking and return at least one byte or an empty string to indicate the end of file. a string or fail The function gets two arguments, the first of which must be a File object f. The second argument must be a positive integer. The function reads data up to len bytes. A string with the result is returned, if everything went well and fail otherwise. In the latter case, one can query the error with .

Note that the reading is done via the buffer of f, such that this function will be quite fast also for large amounts of data.

If the file is already at the end of the file, the function returns a string of length 0. Note that this is not an error!

If a previous call to or to indicated that there is data available to read, then it is guaranteed that the function does not block and returns at least one byte if the file is not yet at end of file and an empty string otherwise. an integer or fail This function can get an arbitrary number of arguments, the first of which must be a File object f. All the other arguments are just written to f if they are strings. Otherwise, the String function is called on them and the result is written out to f.

Note that the writing is done buffered. That is, the data is first written to the buffer and only really written out after the buffer is full or after the user explicitly calls on f.

The result is either the number of bytes written in case of success or fail in case of an error. In the latter case the error can be queried with .

Note that this function blocks until all data is at least written into the buffer and might block until data can be sent again if the buffer is full. an integer or fail Behaves like but works on a single string line and sends an (operating system dependent) end of line string afterwards. Also is called automatically after the operation, such that one can be sure, that the data is actually written out after the function has completed. an integer or fail Behaves like but works on a list of strings list and sends an (operating system dependent) end of line string after each string in the list. Also is called automatically after the operation, such that one can be sure, that the data is actually written out after the function has completed. true or fail This function gets one argument f, which must be a File object. It writes out all the data that is in the write buffer. This is not necessary before the call to the function , since that function calls automatically. However, it is necessary to call after calls to to be sure that the data is really sent out. The function returns true if everything goes well and fail if an error occurs.

Remember that the functions and implicitly call after they are done.

Note that this function might block until all data is actually written to the file descriptor. an integer or fail This function behaves like followed by a call to . It returns either the number of bytes written or fail if an error occurs. true or false This function takes one argument f which must be a File object. It returns true or false according to whether the file f is ready to write. A return value of true guarantees that the next call to on that file will succeed without blocking and accept at least one byte. an integer or fail This function takes four arguments. The first one f must be a File object, the second st a string, and the arguments pos and len must be integers, such that positions pos+1 until pos+len are bound in st. The function tries to write up to len bytes from st from position pos+1 to the file f. If a previous call to or to indicates that f is writable, then it is guaranteed that the following call to will not block and accept at least one byte of data. Note that it is not guaranteed that all len bytes are written. The function returns the number of bytes written or fail if an error occurs. true or false This function takes one argument f which must be a File object. It returns true or false according to whether the file f is ready to flush. A return value of true guarantees that the next call to on that file will succeed without blocking and flush out at least one byte. Note that this does not guarantee, that this call succeeds to flush out the whole content of the buffer! true, false, or fail This function takes one argument f which must be a File object. It tries to write all data in the writing buffer to the file descriptor. If this succeeds, the function returns true and false otherwise. If an error occurs, fail is returned. If a previous call to or indicated that f is flushable, then it is guaranteed that the following call to does not block. However, it is not guaranteed that true is returned from that call. true or fail This function closes the File object f after writing all data in the write buffer out and closing the file descriptor. All buffers are freed. In case of an error, the function returns fail and otherwise true. Note that for pipes to other processes this function collects data about the terminated processes using .
Other functions an integer This function returns the real file descriptor that is behind the File object f. a string or false This function gets one argument f which must be a File object and returns the writing buffer of that File object. This is necessary for File objects, that are not associated to a real file descriptor but just collect everything that was written in their writing buffer. Remember to use this function before closing the File object. an integer or fail This function is the corresponding function to for buffered file access. It behaves similarly to that function. The differences are the following: There are four lists of files r, w, f, and e. They all can contain either integers (standing for file descriptors) or File objects. The list r is for checking, whether files or file descriptors are ready to read, the list w is for checking whether they are ready to write, the list f is for checking whether they are ready to flush, and the list e is for checking whether they have exceptions.

For File objects it is always first checked, whether there is either data available in a reading buffer or space in a writing buffer. If so, they are immediately reported to be ready (this feature makes the list of File objects to test for flushability necessary). For the remaining files and for all specified file descriptors, the function is called to get an overview about readiness. The timeout values t1 and t2 are set to zero for immediate returning if one of the requested buffers were ready.

returns the number of files or file descriptors that are ready to serve or fail if an error occurs. The following function is a convenience function for directory access: a list of strings or fail This function gets a string containing a path name as single argument and returns a list of strings that are the names of the files in that directory, or fail, if an error occurred. true on success and fail on failure Changes the current directory. Returns true on success and fail on failure. The following function is used to create strings describing a pair of an IP address and a port number in a binary way. These strings can be used in connection with the C library functions connect, bind, recvfrom, and sendto for the arguments needing such address pairs. a string This function gets a string ipstring containing an IP address in dot notation, i.e. four numbers in the range from 0 to 255 separated by dots ., and an integer portnr, which is a port number. The result is a string of the correct length to be used for the low level C library functions, wherever IP address port number pairs are needed. The string ipstring can also be a host name, which is then looked up using to find the IP address. a record or fail Takes no arguments, uses to get the environment and returns a record in which the component names are the names of the environment variables and the values are the values. This can then be changed and the changed record can be given to to produce again a list which can be used for as third argument. a list of strings Takes a record as returned by and turns it into a list of strings as needed by as third argument.
Inter process communication fail or the path to an executable If the path name path contains a slash, this function simply checks whether the string path refers to an executable file. If so, path is returned as is. Otherwise, fail is returned. If the path name path does not contain a slash, all directories in the environment variable PATH are searched for an executable with name path. If so, the full path to that executable is returned, otherwise fail.

This function is used whenever one of the following functions gets an argument that should refer to an executable. nothing Closes all file descriptors except those listed in exceptions, which must be a list of integers. a File object or fail The argument path must refer to an executable file in the sense of .

Starts a child process using the executable in path with either stdout or stdin being a pipe. The argument mode must be either the string r or the string w.

In the first case, the standard output of the child process will be the writing end of a pipe. A File object for reading connected to the reading end of the pipe is returned. The standard input and standard error of the child process will be the same than the calling &GAP; process.

In the second case, the standard input of the child process will be the reading end of a pipe. A File object for writing connected to the writing end of the pipe is returned. The standard output and standard error of the child process will be the same than the calling &GAP; process.

In case of an error, fail is returned.

The process will usually die, when the pipe is closed, but can also do so without that. The File object remembers the process ID of the started process and the function then calls for it to acquire information about the terminated process.

Note that activates our SIGCHLD handler (see ).

In either case the File object will have the attribute ProcessID set to the process ID of the child process. a record or fail The argument path must refer to an executable file in the sense of .

A new child process is started using the executable in path The standard input and standard output of it are pipes. The writing end of the input pipe and the reading end of the output pipe are returned as File objects bound to two components stdin and stdout (resp.) of the returned record. This means, you have to write to stdin and read from stdout in the calling &GAP; process. The standard error of the child process will be the same as the one of the calling &GAP; process.

Returns fail if an error occurred.

The process will usually die, when one of the pipes is closed. The File objects remember the process ID of the called process and the function call to for the stdout object will call for it to acquire information about the terminated process.

Note that activates our SIGCHLD handler (see ).

Both File objects will have the attribute ProcessID set to the process ID of the child process, which will also be bound to the pid component of the returned record. a record or fail The argument path must refer to an executable file in the sense of .

A new child process is started using the executable in path The standard input, standard output, and standard error of it are pipes. The writing end of the input pipe, the reading end of the output pipe and the reading end of the error pipe are returned as File objects bound to two components stdin, stdout, and stderr (resp.) of the returned record. This means, you have to write to stdin and read from stdout and stderr in the calling &GAP; process.

Returns fail if an error occurred.

The process will usually die, when one of the pipes is closed. All three File objects will remember the process ID of the newly created process and the call to the function for the stdout object will call for it to acquire information about the terminated child process.

Note that activates our SIGCHLD handler (see ).

All three File objects will have the attribute ProcessID set to the process ID of the child process, which will also be bound to the pid component of the returned record. a record or fail The argument progs is a list of pairs, the first entry being a path to an executable (in the sense of ), the second an argument list, the argument infd is an open file descriptor for reading, outfd is an open file descriptor for writing, both can be replaced by the string open in which case a new pipe will be opened. The argument switcherror is a boolean indicating whether standard error channels are also switched to the corresponding output channels.

This function starts up all processes and connects them with pipes. The input of the first is switched to infd and the output of the last to outfd.

Returns a record with the following components: pids is a list of process ids if everything worked. For each process for which some error occurred the corresponding pid is replaced by fail. The stdin component is equal to false, or to the file descriptor of the writing end of the newly created pipe which is connected to the standard input of the first of the new processes if infd was open. The stdout component is equal to false or to the file descriptor of the reading end of the newly created pipe which is connected to the standard output of the last of the new processes if outfd was open.

Note that the SIGCHLD handler of the `IO` package is installed by this function (see ) and that it lies in the responsibility of the caller to use to ask for the status information of all child processes after their termination. a string or fail Reads the file with the name filename, however, a pipeline is created by the processes described by progs (see ) to filter the content of the file through the pipeline. The result is put into a &GAP; string and returned. If something goes wrong, fail is returned. a string or fail Writes the content of the string st to the file with the name filename, however, a pipeline is created by the processes described by progs (see ) to filter the content of the string through the pipeline. The result is put into the file. If the boolean value append is given and equal to true, then the data will be appended to the already existing file. If something goes wrong, fail is returned. a File object or fail This function is similar to and behaves nearly identically. The only difference is that a filtering pipeline is switched between the file and the File object such that all things read or written respectively are filtered through this pipeline of processes.

The File object remembers the started processes and upon the final call to automatically uses the function to acquire information from the terminated processes in the pipeline after their termination. This means that you do not have to call any more after the call to .

Note that activates our SIGCHLD handler (see ).

The File object will have the attribute ProcessID set to the list of process IDs of the child processes. This functions uses to write the whole string st to the File object f. However, this is done by forking off a child process identical to the calling &GAP; process that does the sending. The calling &GAP; process returns immediately, even before anything has been sent away with the result true. The forked off sender process terminates itself immediately after it has sent all data away.

The reason for having this function available is the following: If one uses or to start up a child process with standard input and standard output being a pipe, then one usually has the problem, that the child process starts reading some data, but then wants to write data, before it received all data coming. If the calling &GAP; process would first try to write all data and only start to read the output of the child process after sending away all data, a deadlock situation would occur. This is avoided with the forking and backgrounding approach.

Remember to close the writing end of the standard input pipe in the calling &GAP; process directly after has returned, because otherwise the child process might not notice that all data has arrived, because the pipe persists! See the file popen2.g in the example directory for an example.

Note that with most modern operating systems the forking off of an identical child process does in fact not mean a duplication of the total main memory used by both processes, because the operating system kernel will use copy on write. However, if a garbage collection happens to become necessary during the sending of the data in the forked off sending process, this might trigger doubled memory usage. a string or fail Starts the process with the executable given by the file name cmd (in the sense of ) with arguments in the argument list args (a list of strings). The standard input and output of the started process are connected via pipes to the calling process. The content of the string input is written to the standard input of the called process and its standard output is read and returned as a string.

All the necessary I/O multiplexing and non-blocking I/O to avoid deadlocks is done in this function.

This function properly does to wait for the termination of the child process but does not restore the original &GAP; SIGCHLD signal handler (see ). a record or fail Starts the process with the executable given by the file name cmd (in the sense of ) with arguments in the argument list args (a list of strings). The standard input, output and error of the started process are connected via pipes to the calling process. The content of the string input is written to the standard input of the called process and its standard output and error are read and returned as a record with components out and err, which are strings.

All the necessary I/O multiplexing and non-blocking I/O to avoid deadlocks is done in this function.

This function properly does to wait for the termination of the child process but does not restore the original &GAP; SIGCHLD signal handler (see ).

The functions returns either fail if an error occurred, or otherwise a record with components out and err which are bound to strings containing the full standard output and standard error of the called process.
Object serialisation (Pickling) The idea of object serialisation is that one wants to store nearly arbitrary &GAP; objects to disk or transfer them over the network. To this end, one wants to convert them to a byte stream that is platform independent and can later be converted back to a copy of the same object in memory, be it in the same &GAP; process or another one maybe even on another machine. The main problem here are the vast amount of different types occurring in &GAP; and the possibly highly self-referential structure of &GAP; objects.

The `IO` package contains a framework to implement object serialisation and implementations for most of the basic data types in &GAP;. The framework is easily extendible to other types and takes complete care of self-references and corresponding problems. It builds upon the buffered I/O functions described in Section . We start by describing the user interface.

Result objects The following static objects are used to report about success or failure of the (un-)pickling operations: This object is returned if an error occurs. This object is returned when there is nothing to return, for example if an unpickler (see ) encounters the end of a file. This object is returned if everything went well and there is no other canonical value to return to indicate this. The only thing you can do with these special values is to compare them to each other and to other objects.

Pickling and unpickling IO&uscore;OK or IO&uscore;Error The argument f must be an open, writable File object. The object ob can be an arbitrary &GAP; object. The operation pickles or serialises the object ob and writes the result into the File object f. If everything is OK, the unique value IO&uscore;OK is returned and otherwise the unique value IO&uscore;Error. The resulting byte stream can be read again using the operation and is platform- and architecture independent. Especially the question whether a system has 32 bit or 64 bit wide words and the question of endianess does not matter.

Note that not all of &GAP;'s object types are supported but it is relatively easy to extend the system. This package supports in particular boolean values, integers, permutations, rational numbers, finite field elements, cyclotomics, strings, polynomials, rational functions, lists, records, compressed vectors and matrices over finite fields (objects are uncompressed in the byte stream but recompressed during unpickling), and straight line programs.

Self-referential objects built from records and lists are handled correctly and are restored completely with the same self-references during unpickling. IO&uscore;Error or a &GAP; object The argument f must be an open, readable File object. The operation reads from f and unpickles the next object. If an error occurs, the unique value IO&uscore;Error is returned. If the File object is at end of file, the value IO&uscore;Nothing is returned. Note that these two values are not picklable, because of their special meaning as return values of this operation here. Nothing This function clears the pickle cache. This cache stores all object pickled in the current recursive call to and is necessary to handle self-references. Usually it is not necessary to call this function explicitly. Only in the rare case (that should not happen) that a pickling or unpickling operation enters a break loop which is left by the user, the pickle cache has to be cleared explicitly using this function for later calls to and to work!
Extending the pickling framework The framework can be extended for other &GAP; object types as follows:

For pickling, a method for the operation has to be installed which does the work. If the object to be pickled has subobjects, then the first action of the method is to call the function IO&uscore;AddToPickled with the object as argument. This will put it into the pickle cache and take care of self-references. Arbitrary subobjects can then be pickled using recursive calls to the operation handing down the same File object into the recursion. The method must either return IO&uscore;Error in case of an error or IO&uscore;OK if everything goes well. Before returning, a method that has called IO&uscore;AddToPickled must call the function IO&uscore;FinalizePickled without arguments under all circumstances. If this call is missing, global data for the pickling procedure becomes corrupt!

Every pickling method must first write a 4 byte magic value such that later during unpickling of the byte stream the right unpickling method can be called (see below). Then it can write arbitrary data, however, this data should be platform- and architecture independent, and it must be possible to unpickle it later without lookahead.

Pickling methods should usually not go into a break loop, because after leaving the user has to call explicitly!

Unpickling is implemented as follows: For every 4 byte magic value there must be a function bound to that value in the record IO&uscore;Unpicklers. If the unpickling operation encounters that magic value, it calls the corresponding unpickling function. This function just gets one File object as argument. Since the magic value is already read, it can immediately start with reading and rebuilding the serialised object in memory. The method has to take care to restore the object including its type completely.

If an object type has subobjects, the unpickling function has to first create a skeleton of the object without its subobjects, then call IO&uscore;AddToUnpickled on this skeleton, before unpickling subobjects. If things are not done in this order, the handling of self-references down in the recursion will not work! An unpickling function that has called IO&uscore;AddToUnpickled at the beginning has to call IO&uscore;FinalizeUnpickled without arguments before returning under all circumstances! If this call is missing, global data for the unpickling procedure becomes corrupt!

Of course, unpickling functions can recursively call to unpickle subobjects. Apart from this, unpickling functions can use arbitrary reading functions on the File object. However, they should only read sequentially and never move the current file position pointer otherwise. An unpickling function should return the newly created object or the value IO&uscore;Error if an error occurred. They should never go into a break loop, because after leaving the user has to call explicitly!

Perhaps the best way to learn how to extend the framework is to study the code for the basic &GAP; objects in the file pkg/io/gap/pickle.gi.
Really random sources This section describes so called real random sources. It is an extension to the library mechanism of random source objects that uses the devices /dev/random and /dev/urandom available on Linux systems (and maybe on other operating systems) providing random numbers that are impossible to predict. The idea is that such sources of random numbers are useful to produce unpredictable secret keys for cryptographic applications.
The functions a real random source object or fail The first argument r must be the &GAP; filter IsRealRandomSource and the second either the string random or the string urandom. A real random source object is created that draws its random numbers from the kernel devices /dev/random and /dev/urandom respectively. Whereas /dev/urandom always provides random numbers of not guaranteed quality, the device /dev/random measures its entropy and produces guaranteed unpredictable numbers. However, it might block until enough random events (like mouse movements) have been accumulated.
A client side implementation of the HTTP protocol The `IO` package contains an implementation of the client side of the HTTP protocol. The basic purpose of this is of course to be able to download data from web servers from the &GAP; language. However, the HTTP protocol can perform a much bigger variety of tasks.
Functions for client side HTTP a record The first argument hostname must be a string containing the hostname of the server to connect. The second argument port must be an integer in the range from 1 to 65535 and describes the port to connect to on the server.

The function opens a TCP/IP connection to the server and returns a record conn with the following components: conn.sock is fail if an error occurs and otherwise a File object linked to the file descriptor of the socket. In case of an error, the component conn.errormsg contains an error message, it is otherwise empty. If everything went well then the component conn.host is the result from the host name lookup (see ) and the component conn.closed is set to false.

No data is sent or received on the socket in this function. a record This function performs a complete HTTP request. The first argument must be a connection record as returned by a successful call to . The argument method must be a valid HTTP request method in form of a string. The most common will be GET, POST, or HEAD. The argument uri is a string containing the URI of the request, which is given in the first line of the request. This will usually be a relative or absolute path name given to the server. The argument header must be a &GAP; record. Each bound field of this record will we transformed into one header line with the name of the component being the key and the value the value. All bound values must be strings. The argument body must either be a string or false. If it is a string, this string is sent away as the body of the request. If no string or an empty string is given, no body will be sent. The header field Content-Length is automatically created from the length of the string body. Finally, the argument target can either be false or a string. In the latter case, the body of the request answer is written to the file with the name given in target. The body component of the result will be the file name in this case. If target is false, the full body of the answer is stored into the body component of the result.

The function sends away the request and awaits the answer. If anything goes wrong during the transfer (for example if the connection is broken prematurely), then the component statuscode of the resulting record is 0 and the component status is a corresponding error message. In that case, all other fields may or may not be bound to sensible values, according to when the error occurred. If everything goes well, then statuscode and status are bound to the corresponding values coming from the request answer. statuscode is transformed into a &GAP; integer. The header of the answer is parsed, transformed into a &GAP; record, and stored into the component header of the result. The body component of the result record is set as described above. Finally, the protoversion component contains the HTTP protocol version number used by the server as a string and the boolean value closed indicates, whether or not the function has detected, that the connection has been closed by the server. Note that by default, the connection will stay open, at least for a certain time after the end of the request.

See the description of the global variable for rules how timeouts are done in this function.

Note that if the method is HEAD, then no body is expected (none will be sent anyway) and the function returns immediately with empty body. Of course, the Content-Length value in the header is as if it the request would be done with the GET method. This global variable holds a list of length two. By default, both entries are fail indicating that should never timeout and wait forever for an answer. Actually, the two values in this variable are given to the function call during I/O multiplexing. That is, the first number is in seconds and the second in milliseconds. Together they lead to a timeout for the HTTP request. If a timeout occurs, an error condition is triggered which returns a record with status code 0 and status being the timeout error message.

You can change the timeout by accessing the two entries of this write protected list variable directly. nothing Closes the connection described by the connection record conn. No error can possibly occur. a record The arguments are as the corresponding ones in the functions and respectively. This function opens an HTTP connection, tries a single HTTP request and immediately closes the connection again. The result is as for the function. If an error occurs during the opening of the connection, the statuscode value of the result is 0 and the error message is stored in the status component of the result. The previous function allows for a very simple implementation of a function that checks, whether your current &GAP; installation is up to date: nothing This function accesses a web page in St. Andrews and runs some &GAP; code from there. This code knows all the currently released versions of &GAP; and its packages. It prints out a summary and possibly suggests upgrades. If you do not want to executed code downloaded from the internet, then do not call this function.

More concretely, the page accessed is http://www.gap-system.org/Download/upgrade.html and the code executed is a single call to the function SuggestUpgrades function in the &GAP; library. nothing This function downloads the file from the given uniform resource locator URL using the HTTP protocol and reads the contents into &GAP; using .

Note that this can execute arbitrary code on your machine with the privileges of the &GAP; job running, so you should be very careful what files you download and execute. You have been warned!
Background jobs using fork This chapter describes a way to use multi-processor or multi-core machines from within &GAP;. In its current version the &GAP; system is a single threaded and single process system. However, modern operating systems allow, via the fork system call, to replicate a complete process on the same machine relatively efficiently. That is, at first after a fork the two processes actually use the same physical memory such that not much copying needs to be done. The child process is in exactly the same state as the parent process, sharing open files, network connections and the complete status of the workspace. However, whenever a page of memory is written, it is then automatically copied using new, additional physical memory, such that it behaves like a completely separate process. This method is called copy-on-write.

Thus this is a method to parallelise certain computations. Note however, that from the point of time when the fork has occurred, all further communication between the two processes has to be realised via pipes or even files.

The operations and methods described in this chapter help to use &GAP; in this way and implement certain skeletons of parallel programming to make these readily available in &GAP;. Note that this implementation has its severe limitations and should probably eventually be replaced by a proper multi-threaded version of &GAP;.
Background jobs One creates a background job with the following operation: a background job object or fail This operation creates a background job using which starts up as an identical copy of the currently running &GAP; process. In this child process the function fun is called with the argument list args. The third argument opt must be a record for options. The operation returns either an object representing the background job or fail if the startup did not work.

This operation automatically sets up two pipes for communication with the child process. This is in particular used to report the result of the function call to fun back to the parent. However, if called without the option TerminateImmediately (see below) the child process stays alive even after the completion of fun and one can submit further argument lists for subsequent calls to fun. Of course, these additional argument lists will have to be sent over a pipe to the child process. A special case is if the argument args is equal to fail, in this case the child process is started but does not automatically call fun but rather waits in an idle state until an argument list is submitted via the pipe using the operation described below.

There are two components defined which can be bound in the options record opt. One is TerminateImmediately, if this is bound to true then the child process immediately terminates after the function fun returns its result. In this case, no pipe for communication from parent to child is created since it would never be used. Note that in this case one can still get the result of the function fun using the operation described below, even when the child has already terminated, since the result is first transmitted back to the parent before termination. The following operations are available to deal with background job objects: true, false or fail This operation checks whether or not the background job represented by the object job has already finished the function call to its worker function and is now idle. If so, true is returned. If it is still running and working on the worker function, false is returned. If the background job has already terminated altogether, this operation returns fail. Note that if a child process terminates automatically after the first completion of its worker function and sending the result, then the first call to after completion will return true to indicate successful completion and all subsequent calls will return fail. true or false This operation checks whether or not the background job represented by the object job has already terminated. If so, true is returned, if not, false is returned. the result of the worker function or fail This operation waits until the worker function of the background job job has finished and the job is idle. It then returns the result of the worker function, which has automatically been transmitted to the parent process. If the child process has died before completion fail is returned. the result of the worker function or fail This operation does the same as . true or fail This submits another argument list args for another call to the worker function in the background job job. It is an error if either the background job has already terminated or if it is still busy working on the previous argument list. That is, one must only submit another argument in a situation when would return true. This is for example the case directly after a successful call to or i which did not return fail, unless the background job was created with the TerminateImmediately option set to true.

This operation returns immediately after submission, when the new argument list has been sent to the child process through a pipe. In particular, it does not await completion of the worker function for the new argument list. nothing This kills the background job represented by the object job with immediate effect. No more results can be expected from it. Note that unless one has created the background job with the TerminateImmediately option set to true one always has to call on a background job eventually for cleanup purposes. Otherwise, the background job and the connecting pipes remain alive until the parent &GAP; process terminates.
Parallel programming skeletons In this section we document the operations for the available skeletons. For a general description of these ideas see other sources. a list of results or fail The argument jobs must be a list of &GAP; functions and the argument args a list of the same length containing argument lists with which the job functions can be called. This operation starts up a background job using fork for each of the functions in jobs, calls it with the corresponding argument list in args. As soon as any of the background jobs finishes with a result, terminates all other jobs and reports the results found so far. Note that it can happen that two jobs finish at the same time in the sense that both results are received before all other jobs could be terminated. Therefore the result of is a list, in which position i is bound if and only if job number i returned a result. So in the result at least one entry is bound but it is possible that more than one entry is bound.

You can specify an overall timeout to give up the whole computation if no job finishes by setting the TimeOut component of the options record opt. In this case you have to set it to a record with two components tv_sec and tv_usec which are seconds and microseconds respectively, exactly as returned by the function. In the case of timeout an empty list is returned. a list of results or fail The argument jobs must be a list of &GAP; functions and the argument args a list of the same length containing argument lists with which the job functions can be called. This operation starts up a background job using fork for each of the functions in jobs, calls it with the corresponding argument list in args. As soon as all of the background jobs finish with a result, reports the results found. Therefore the result of is a list, in which position i is bound to the result that job number i returned.

You can specify an overall timeout to stop the whole computation if not all jobs finish in time by setting the TimeOut component of the options record opt. In this case you have to set it to a record with two components tv_sec and tv_usec which are seconds and microseconds respectively, exactly as returned by the function. In the case of timeout a list is returned in which the positions corresponding to those jobs that have already finished are bound to the respective results and the other positions are unbound. a list of results or fail This is a parallel version of the function. It applies the function worker to all elements of the list l and returns a list containing the results in corresponding positions. You have to specify the component NumberJobs in the options record opt which indicates how many background processes to start. You can optionally use the TimeOut option exactly as for , however, if a timeout occurs, returns fail.

Note that the usefulness of this operation is relatively limited, since every individual result has to be sent back over a pipe from the child process to the parent process. Therefore this only makes sense if the computation time for the worker function dominates the communication time. a value or fail This is a parallel version implementation of the classical MapReduce pattern. It applies the function map to all elements of the list l and then reduces the result using the reduce function which accepts two return values of map and returns one of them. Thus, the final result is one return value or fail if the startup of the jobs fails. You have to specify the component NumberJobs in the options record opt which indicates how many background processes to start. You can optionally use the TimeOut option exactly as for , however, if a timeout occurs, returns fail.

Note that this can be very useful because quite often the cumulated computation time for all the worker function calls dominates the communication time for a single result. Note that the next parallel skeleton is a worker farm which is described in the following section.
Worker farms The parallel skeleton of a worker farm is basically nothing but a bunch of background jobs all with the same worker function and all eagerly waiting for work. The only additional concepts needed are an input and an output queue. The input queue contains argument lists and the output queue pairs of argument lists and results.

One creates a worker farm with the following operation: an object representing the worker farm or fail This operation creates a worker farm with the worker function fun and sets up its input and output queue. An object representing the farm is returned unless not all jobs could be started up in which case fail is returned. After startup all background jobs in the farm are idle. The only valid option in the options record opt is NumberJobs and it must be bound to the number of worker jobs in the farm, a positive integer. The following operations are for worker farm objects: nothing This operation called on a worker farm object wf administrates the input and output queues of the worker farm. In particular it checks whether new results are available from the workers and if so it appends them to the output queue. If jobs are idle and the input queue is non-empty, argument lists from the input queue are sent to the idle jobs and removed from the input queue.

This operation must be called regularly to keep up the communication with the clients. It uses select and so does not block if the boolean argument block is set to false. However, if larger chunks of data has to be sent or received this operation might need some time to return.

If the boolean argument block is set to true then the blocks until at least one job has returned a result. This can be used to wait for the termination of all tasks without burning CPU cycles in the parent job. One would repeatedly call with block set to true and after each such call check with whether all tasks are done. Note that one should no longer call with block set to true once this is the case since then it would block forever.

This operation is called automatically by most of the following operations. nothing This operation terminates all background jobs in the farm wf, which cannot be used subsequently. One should always call this operation when the worker farm is no longer needed to free resources. true or false This operation returns true if all background jobs in the worker farm wf are idle. This means, that all tasks which have previously been submitted using have been completed and their result been appended to the output queue. The operation is automatically called before the execution of . nothing This operation submits a task in the form of an argument list for the worker function to the worker farm. It is appended at the end of the input queue. The operation is automatically called after the execution of , giving the farm a chance to actually send the work out to the worker background jobs. nothing This operation collects all results from the output queue of the worker farm. The output queue is empty after this function returns. The results are reported as a list of pairs, each pair has the input argument list as first component and the output object as second component.

The operation is automatically called before the execution of , giving the farm a chance to actually receive some more results from the worker background jobs.
I/O multiplexing
Introduction Whenever one needs to do input/output on more than one connection (file descriptor) at a time, some code is needed to organise the I/O multiplexing. Due to the single-threaded nature of the current &GAP; language one has to use and some buffering and queueing to organise this. This chapter describes a relative generic implementation of I/O-multiplexing using so-called objects. The basic idea is that an object handles lots of I/O connections at the same time and maintains a buffer for each of them. There is a very simple protocol that marks chunks of data (called messages) and whenever a message has been received completely it is collected in the input queue of the , marked with the number of the connection it came from. Rather than sending a message away in one go, one would always schedule it for sending by appending it to the output queue. The operation , when called often enough, will then make sure that the message is sent away eventually.
The operations for IOHub objects In this section, we simply describe the functions and operations to create, use and destroy objects. an object This creates a new object at first without any open connections. a positive integer This operation adds a new connection to the object h. The arguments i and o must be Unix file descriptors or 0 and i must be open for reading if it is positive and o must be open for writing if it is positive. It is allowed that both file descriptors are equal, but they may not both be equal to 0. The operation returns a positive integer which is the number under which this new connection will be administrated in the object. Note that this number is specific to the object h.

From the moment these file descriptors are registered with the object, every subsequent call to will try to do input and output on them. This means in particular that the other side of this connection should be in the same initial state of the protocol. Usually this will be achieved by them being added as a new connection to a corresponding object on the other side at the same time.

See also below. nothing The argument h must be an object and nr the number of a connection which was previously returned by . The corresponding connection is closed and removed from the . a Unix file descriptor or fail The argument h must be an object, addr an IP address or host name as a string and port a port number (see also ). This operation creates a new socket, binds it to the IP address and port and attaches it to the object. From this moment on the operation will accept new bidirectional TCP/IP connections on that socket and add them to h. The operation returns either the file descriptor of the new socket or fail if an error occurred. nothing The argument h must be an object. Any server socket which was attached to h is shut down, so no new connections will be accepted. nothing The argument h must be an object. All connections of h will be closed using and any serving socket will be shut down using . The object will not be usable any more after this call. a positive integer or fail The argument h must be an object. The object h must have a serving socket attached to it via , otherwise fail is returned and nothing happens. One more connection is accepted through the serving socket. It is added as a new bidirectional TCP/IP connection to the object and the new connection number is returned. Note first that this operation blocks until a new connection comes in. Note furthermore that this operation is usually called automatically in whenever a new connection has come in, which is reported in the internal call. So usually, the client code does not have to call this operation at all. true or fail The argument h must be an object, nr must be a positive integer which is the number of an open connection of h which can be used for output. The argument st must be a &GAP; string. This operation appends the message st to the end of the output queue for the connection nr. Note that at this stage no output is actually performed automatically. One has to call subsequently to actually send the message away. a list of length 2 The argument h must be an object, nr must be an integer. If nr is positive, this operation returns the earliest message which has come in from connection number nr and has not yet been returned by before. This message is then removed from the input queue. If there is no such message, then false is returned. A message is returned as a plain list of length 2 where the first entry is the connection number it came from and the second entry is a string containing the message itself. If nr is equal to 0 then the first message in the input queue from any connection is returned or false if there is no message in the input queue. a connection number or fail The argument h must be an object, the arguments addr and port must be an address/port pair as used in , so address can either be a host name or an IP address and port is a port number. This operation opens a new TCP connection to the address and port specified, adds a new bidirectional connection to the h using and returns the connection number specific to the object h. If anything goes wrong, fail is returned. a list The argument h must be an object. This returns the internal object for the output queue. Its elements are pairs where the first entry is the connection number where it is going to be sent and the second entry is the message as a string. Only modify this list if you really know what you are doing. a list The argument h must be an object. This returns the internal object for the input queue. Its elements are pairs where the first entry is the connection number from where the message was received and the second entry is the message as a string. Only modify this list if you really know what you are doing. true or false or fail The argument h must be an object, and the optional second argument block must be true or false. This operation uses to decide which of the file descriptors belonging to the connections of h are ready to read or write. All file descriptors which are ready are served, possibly updating the input and output queues. A possible serving socket is also served accepting a new connection if there is one. The operation loops until no more file descriptors are ready. It returns true if some I/O was performed and false if not. It returns fail if the is already shut down. The second argument block indicates whether or not should block until some I/O has taken place. If this argument is omitted then false (non-blocking operation) is the default.

Note that broken connections are silently closed.
Examples There is an example hash server in the file examples/hashserver.g.
Examples of usage For larger examples see the example directory of the package. You find there a small server using the TCP/IP protocol and a corresponding client and another small server using the UDP protocol and a corresponding client.

Further, there is an example for the usage of File objects, that read from or write to strings.

Another example there shows starting up a child process and piping a few megabytes through it using .

In the following, we present a few explicit, interactive short examples for the usage of the functions in this package. Note that you have to load the `IO` package with the command LoadPackage("IO"); before trying these examples.
Writing and reading a file The following sequence of commands opens a file with name guck and writes some things to it: gap> f := IO_File("guck","w"); <file fd=3 wbufsize=65536 wdata=0> gap> IO_Write(f,"Hello world\n"); 12 gap> IO_WriteLine(f,"Hello world2!"); 14 gap> IO_Write(f,12345); 5 gap> IO_Flush(f); true gap> IO_Close(f); true There is nothing special about this, the numbers are numbers of bytes written. Note that only after the command the data is actually written to disk. Before that, it resides in the write buffer of the file. Note further, that the call here would not have been necessary, since the call flushes the buffer anyway.

The file can again be read with the following sequence of commands: gap> f := IO_File("guck","r"); <file fd=3 rbufsize=65536 rpos=1 rdata=0> gap> IO_Read(f,10); "Hello worl" gap> IO_ReadLine(f); "d\n" gap> IO_ReadLine(f); "Hello world2!\n" gap> IO_ReadLine(f); "12345" gap> IO_ReadLine(f); "" gap> IO_Close(f); true Note here that reading line-wise can only be done efficiently by using buffered I/O. You can mix calls to and to . The end of file is indicated by an empty string returned by one of the read functions.
Using filtering programs to read and write files If you want to write a big amount of data to file you might want to compress it on the fly without using much disk space. This can be achieved with the following command: gap> s := "";; for i in [1..10000] do Append(s,String(i)); od;; gap> Length(s); 38894 gap> IO_FileFilterString("guck.gz",[["gzip",["-9c"]]],s); true gap> sgz := StringFile("guck.gz");; gap> Length(sgz); 18541 gap> ss := IO_StringFilterFile([["gzip",["-dc"]]],"guck.gz");; gap> s=ss; true This sequence of commands needs that the program gzip is installed on your system.
Using filters when reading or writing files sequentially If you want to process bigger amounts of data you might not want to store all of it in a single &GAP; string. In that case you might want to access a file on disk sequentially through a filter: gap> f := IO_FilteredFile([["gzip",["-9c"]]],"guck.gz","w"); <file fd=5 wbufsize=65536 wdata=0> gap> IO_Write(f,"Hello world!\n"); 13 gap> IO_Write(f,Elements(SymmetricGroup(5)),"\n"); 1359 gap> IO_Close(f); true gap> f := IO_FilteredFile([["gzip",["-dc"]]],"guck.gz","r"); <file fd=4 rbufsize=65536 rpos=1 rdata=0> gap> IO_ReadLine(f); "Hello world!\n" gap> s := IO_ReadLine(f);; Length(s); 1359 gap> IO_Read(f,10); "" gap> IO_Close(f); true
Accessing a web page The `IO` package has an HTTP client implementation. Using this you can access web pages and other web downloads from within &GAP;. Here is an example: gap> r := SingleHTTPRequest("www.math.rwth-aachen.de",80,"GET", > "/~Max.Neunhoeffer/index.html",rec(),false,false);; gap> RecFields(r); [ "protoversion", "statuscode", "status", "header", "body", "closed" ] gap> r.status; "OK" gap> r.statuscode; 200 gap> r.header; rec( date := "Thu, 07 Dec 2006 22:08:22 GMT", server := "Apache/2.0.55 (Ubuntu)", last-modified := "Thu, 16 Nov 2006 00:21:44 GMT", etag := "\"2179cf-11a5-3c77f600\"", accept-ranges := "bytes", content-length := "4517", content-type := "text/html; charset=ISO-8859-1" ) gap> Length(r.body); 4517 Of course, the time stamps and exact sizes of the answer may differ when you do this.
(Un-)Pickling Assume you have some &GAP; objects you want to archive to disk grouped together. Then you might do the following: gap> r := rec( a := 1, b := "Max", c := [1,2,3] ); rec( a := 1, b := "Max", c := [ 1, 2, 3 ] ) gap> r.c[4] := r; rec( a := 1, b := "Max", c := [ 1, 2, 3, ~ ] ) gap> f := IO_File("guck","w"); <file fd=3 wbufsize=65536 wdata=0> gap> IO_Pickle(f,r); IO_OK gap> IO_Pickle(f,[(1,2,3,4),(3,4)]); IO_OK gap> IO_Close(f); true Then, to read it in again, just do: gap> f := IO_File("guck"); <file fd=3 rbufsize=65536 rpos=1 rdata=0> gap> IO_Unpickle(f); rec( a := 1, b := "Max", c := [ 1, 2, 3, ~ ] ) gap> IO_Unpickle(f); [ (1,2,3,4), (3,4) ] gap> IO_Unpickle(f); IO_Nothing gap> IO_Close(f); true Note that this works for a certain amount of builtin objects. If you want to archive your own objects or more sophisticated objects you have to use extend the functionality as explained in Section . However, it works for lists and records and they may be arbitrarily self-referential.
License This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.