Using shared libraries in OpenMCL


Table of Contents

Overview
Functional Reference
open-shared-library [Function]
close-shared-library [Function]
external [Macro]
Examples
Limitations and known bugs
Darwin Notes

Overview

OpenMCL provides facilities to open and close shared libraries.

"Opening" a shared library maps the library's code and data into OpenMCL's address space and makes its exported symbols accessible to OpenMCL.

"Closing" a shared library unmaps the library's code and data and removes the library's symbols from the global namespace.

A small number of shared libraries (including libc, libm, libdl under Linux, and the "system" library under Darwin) are opened by the lisp kernel and can't be closed.

OpenMCL uses data structures of type EXTERNAL-ENTRY-POINT to map a foreign function name (string) to that foreign function's current< address. (A function's address may vary from session to session as different versions of shared libraries may load at different addresses; it may vary within a session for similar reasons.)

An EXTERNAL-ENTRY-POINT whose address is known is said to be resolved. When an external entry point is resolved, the shared library which defines that entry point is noted; when a shared library is closed, the entry points that it defines are made unresolved. An EXTERNAL-ENTRY-POINT must be in the resolved state in order to be FF-CALLed; calling an unresolved entry point causes a "last chance" attempt to resolve it. Attempting to resolve an entrypoint that was defined in a closed library will cause an attempt to reopen that library.

OpenMCL keeps track of all libraries that have been opened in a lisp session. When a saved application is first started, an attempt is made to reopen all libraries that were open when the image was saved, and an attempt is made to resolve all entrypoints that had been referenced when the image was saved. Either of these attempts can fail "quietly", leaving some entry points in an unresolved state.

Linux shared libraries can be referred to either by a string which describes their full pathname or by their soname, a shorter string that can be defined when the library is created. The dynamic linker mechanisms used in Linux make it possible (through a series of filesystem links and other means) to refer to a library via several names; the library's soname is often the most appropriate identifier.

sonames are often less version-specific than other names for libraries; a program that refers to a library by the name "libc.so.6" is more portable than one which refers to "libc-2.1.3.so" or to "libc-2.2.3.so", even though the latter two names might each be platform-specific aliases of the first.

All of the global symbols described below are exported from the CCL package.

Functional Reference

open-shared-library [Function]

Syntax

open-shared-library name

Description

If the library denoted by name can be loaded by the operating system, returns an object of type SHLIB that describes the library; if the library is already open, increments a reference count. If the library can't be loaded, signals a SIMPLE-ERROR which contains an often cryptic message from the operating system.

Arguments

name

A SIMPLE-STRING which is presumed to be the soname of or a filesystem path to the library.

close-shared-library [Function]

Syntax close-shared-library shlib &key (completely t)

Description Decrements the shared library's reference count until it becomes 0 (if completely is T) or by 1 otherwise. If the reference count becomes 0, frees memory resources consumed by the library and causes any EXTERNAL-ENTRY-POINTs known to be defined in the library to become unresolved.

Arguments

shlib Either a SHLIB object as returned by OPEN-SHARED-LIBRARY or a string which denotes the soname of some open library.

completely A boolean which indicates whether the library's reference count should be decremented to 0 or by 1

external [Macro]

Syntax

external name

Description

Finds or creates an EXTERNAL-ENTRY-POINT which maintains the address of specified foreign symbol. Tries to resolve the entry point and identify the containing library.

Arguments

 

name

A simple string, presumably the name of a foreign function. Case is significant.

Examples

;;; Try to do something simple.
? (open-shared-library "libgtk.so")
> Error: Error opening shared library "libgtk.so": /usr/lib/libgtk.so: undefined symbol: gdk_threads_mutex
> While executing: OPEN-SHARED-LIBRARY
;;; Grovel around, curse, and try to find out where "gdk_threads_mutex"
;;; might be defined. Then try again:
? (open-shared-library "libgdk.so")
#<SHLIB libgdk.so #x3046DBB6>
? (open-shared-library "libgtk.so")
#<SHLIB libgtk.so #x3046DC86>
;;; Reference an external symbol defined in one of those libraries.
? (external "gtk_main")
#<EXTERNAL-ENTRY-POINT "gtk_main" (#x012C3004) libgtk.so #x3046FE46>
;;; Close those libraries.
? (close-shared-library "libgtk.so")
T
? (close-shared-library "libgdk.so")
T
;;; Reference the external symbol again.
? (external "gtk_main")
#<EXTERNAL-ENTRY-POINT "gtk_main" {unresolved} libgtk.so #x3046FE46>
  

Limitations and known bugs

  • Don't get me started.

  • The underlying functionality has a poor notion of dependency; it's not always possible to open libraries that depend on unopened libraries, but it's possible to close libraries on which other libraries depend.

    It may be possible to generate more explicit dependency information by parsing the output of the Linux ldd and ldconfig programs.

Darwin Notes

Darwin shared libraries come in two (basic) flavors:

  • "dylibs" (which often have the extension ".dylib") are primarily intended to be linked against at compile/link time. They can be loaded dynamically, but can't be unloaded. Accordingly, OPEN-SHARED-LIBRARY can be used to open a .dylib-style library; calling CLOSE-SHARED-LIBRARY on the result of such a call has no effect.

    It appears that (due to an OS bug) attempts to open .dylib shared-libraries that are already open can cause memory corruption unless the full pathname of the .dylib file is specified on the first and all subsequent calls.

  • "bundles" are intended to serve as application extensions; they can be opened multiple times (creating multiple instances of the library!) and closed properly.

Thanks to Michael Klingbeil for getting both kinds of Darwin shared libraries working in OpenMCL.