lib.pl -- Predicate based code development.

This pack implements methods for loading code into SWI Prolog programs.

Main innovations

Lazy loading

One of the major innovations the library introduces, is that of progressive, lazy loading of packs. That is, if only a specific predicate is (lazily) required from a pack(lib)-aware pack, only that and its dependent code will be loaded.

That is, your code can load things like

?- lib( stoics_lib:kv_compose/3 ).
?- lib( stoics_lib:kv_decompose/3 ).

and only the relevant parts of the pack(stoics_lib) will be loaded.

If later on your code decides to do a

?- lib(stoics_lib).

The remainder of the library loads up quietly and politely.

Please note that this is, at top level at least, orthogonal to any other loading.

You can still do

?- use_module( library(stoics_lib) ).

and get the whole thing into memory.

A good example of how to create a lazy pack is pack(stoics_lib), http://stoics.org.uk/~nicos/sware/stoics_lib v0.3. An example of how to lazy load things from stoics_lib is the latest pack(debug_call) http://stoics.org.uk/~nicos/sware/debug_call v0.4.

Cells

As of version 2.0 the pack supports hierarchical module de-composition.

A cell compose pack, is build by a skeleton module that all cells depend on and then a number of independent cells that can be loaded independently as well as in combination.

There are at least 2 reasons why one would like decomposable modules: (a) resources, and (b) clarity of interface. Only loading parts of a module can result in smaller memory consumption as irrelevant bits are not loaded. Also, if modules have long lists of defined predicates, like bio_db v2.0, then loading only conceptually clear sub-set of a module allows programmer to focus on the predicates that are relevant to a specific task.

pack(bio_db) was the driving force for developing cell based packs and it provides natural cell units. At the top level there are two cells, hs for human biological data and mouse for mouse data. Each cell is further broken to a number of cells each corresponding to the source database where data is converted from. For instance hs contains sub-cells: ense, gont, hgnc, ncbi, pros, strg and unip.

See pack(bio_db/cell/hs.pl) and pack(bio_db/cell/mouse.pl).

Cell based pack can still be viewed and loaded as normal module files. For instance,

?- use_module(library(bio_db)).

Loads the whole interface (all cells), without the user needing to be aware of anything. The only difference is that the user will not be able to see all the module predicates at the first line of file pack(bio_db/prolog/bio_db.pl)).

?- lib(bio_db).

Also loads everything.

?- lib(& bio_db).

Loads the skeleton of the module (cells usually laod the module dependencies like this).

?- lib(& bio_db(hs)).

Loads hs cell, which in this case comprises of number of sub-cells.

?- lib(& bio_db(hs)).

Loads hs cell (and skeleton). hs comprises of a number of sub-cells.

?- lib(& bio_db(hs(hgnc))).

Loads the hs/hgnc primary cell (and the skeleton).

?- use_module( pack('bio_db/cell/hs/hgnc') ).

?- lib(@ bio_db)

Loads all sub-cells of a library.

?- load_files( library(bio_db) ).

Will load everything even if cell based loading ahs taken place. (use_module(library(bio_db)) would work.)

---+++ Suggested code

The library supports _suggested_ loading and code execution. These operations are meant for fringe features that
are not, by default reported if missing. Reporting in form of warnings can be turned on by either setting flag
_lib_suggests_warns_ to true (globally controlled), or passing option (local, controlled by developer).

Prolog lag _lib_suggests_warns_ can take values:
  * auto
    (default flag value), silent by default unless loading code presents option suggests_warns(true)
  * false
    never warn when suggested features are missing
  * true
    always warn when features are missing.

:- lib(suggests(wgraph),[]). :- lib(real). :- lib(suggests(call(lib_r("GGally"),[]).


---+ Other features

---+++ General points

Pack(lib) plays reasonably well with the documentation server. Bar, the normal
limitations of the server.  By convention and to help locating the module docs,
lazy packs should define (Pack)/0 predicate in same file as the mods docs.
Searching for that on doc server, should make it easy enough to get to it.

 Although this library, _pack(lib)_, contains a number of involved features
it can also be used as a straight forward shorthand,
replacement for use_module(library(Lib)).

 ==
 ?- lib(Atomic).
 ==
 is equivelant to =|use_module(library(Atomic))|= if Atomic is a system library
 or an installed Pack, while it will interogate the SWI pack server
 for matching packs if Atomic is atomic and not an installed pack.

In addition the library allows for loading with initializations turned off.



---+++ Repositories

Code is managed in _repositories_ (also _repo_) that can be either packs or libs (ie local directories).

A _pack_ is a unit of programs as managed by built-in SWI package manager
(=|library(prolog_pack)|=). A _lib_ (_library_) is a directory containing a
number of program files.

pack(lib) supports a number of ways to organise your code and load it, but
it comes to its own when code s organised as predicate-in-a-file fashion.
In this mode of development a predicate such as kv_decompose/3
would be defined on file kv_decompose.pl which will only containing code for defining
this predicate, with the possible exception of helper predicates that are
too specific to be of outside interest.

kv_decompose( [], [], [] ). kv_decompose( [K-V|T], [K|Tk], [V|Tv] ) :- kv_decompose( T, Tk, Tv ).


Lib code is considered as coming from the special pack _user_.

---+++ Code-tables

Associated with each repository are 2 types of code-tables: (code-)_indices_ and (file-)_locators_.

A code-index maps a predicate identifier along with its source repo to
an absolute file name of the source that defines it.
Indices are of the form:

lib_tables:lib_index( Pname, Parity, Repo, AbsFile ).


File locators store all filenames in a repository.
These can be named matched predicate names that need to be loaded.
pack(lib) can be directed to assume that files from a specific
repository exhibit this _homonyms_ property.
Locators are of the form:

lib_tables:lib_homonym( Stem, Repo, AbsFile ).


For each index file loaded for a repository, the following is asserted:

lib_tables:lib_loaded_indices(Repo,File)


and for each locator

lib_tables:lib_loaded_homonyms(Repo,Stem,File)


When loading a repository the user can choose whether to load
indices and locators independently.

---+++ Loading source code

During the process of loading code into memory, lib/1 and /2 directives are used
to locate code to which the specific code depends.

There are three main categories of operations:
  * load code from explicitly identified repository
  * attach a repository for implicit loading
  * load code from attached repositories

These operations are all specific to the loading context. This is achieved by
creating meta-predicates that identify which part of the repository base each
context has access to.

Attachment of repository is registered via =|lib_tables:lib_attached_indices(To,PackIG)|=
and =|lib_tables:lib_attached_homonyms(To,PackFG)|=.

lib_tables:lib_attached_indices(bims,options). lib_tables:lib_attached_homonyms(bims,false).

attaches the indices but not the file locators.

Since all code from directory-libs load to a single module (_user_), loading
code has either access to all such code, or to none.

---+++ Conventions

Packs are expected to have matching top directories and main files. The main file
of a pack should be within top directory _prolog/_. (The directory convention
is set by library(prolog_pack)). For example for pack _bims_ the following file should exist
in packs directory:

bims/prolog/bims.pl ==

For packs the main code directory is src/. Additionally src/lib and src/auxil are treated as code directories.

Internals

Variables

Repo: repository
Pack: prolog pack (installed or locally addressed)
Lib: directory containing code
Root: absolute reference to the root directory of a Repo
Pn: predicate name
Idx: library index term
Hmn: library file homonym term
Pa: predicate arity
Cxt: context module

Predicate names

index
file

Pack info

This is a complete re-write of pack(requires) v1.1.

Listens to debug(lib).

author: - nicos angelopoulos
version: - 1.0 2017/3/6; - 1.1 2017/3/9, lazy loading; - 1.2 2017/3/11, fixed missing cut, added lib(version(V,D)); - 1.3+4 2017/8/8, fixed multi-source for user, improved contact to server, install while lazy loading; - 1.5 2017/8/15; - 1.6 2018/3/18, lib/2 suggests(), lib/2, promise() via hot-swapping, private packs; - 1.7 2018/4/5, auto-install missing was broken; - 2.2 2018/11/26, cell based module compositionality, & operator (by default load everything); - 2.3 2019/4/18, lib_code_loader/3 hook & lib_r/2, suggests failure messages via lib_suggests_warns flag & options; - 2.4 2019/4/22, small fix release; - 2.5 2019/5/8, bioc (for bioconductor) load term
See also: - http://stoics.org.uk/~nicos/sware/lib

lib(+Operand)

lib(+Operand, +Opts)

Loads code or/and indices of Repo into the current context.

When Repo homonym(Repository) then only the homonims of local dir (adjusted for pack dir structure) are added to as coming from Repository.

Operands

SysLibrary

An installed library (atomic). Is loaded with use_module( library(SysLibrary) ).

Pack

A pack known to SWI. If pack is not installed then the server is contacted to look for name-matching packs that can be installed. If there is at least one matching pack, it can be installed interactively.

LibDir

Declares a library directory should be

atomic, an absolute path
rel(Rel) Rel will be absolute_file_name/3 to be made into an absolute location
alias(Dir) as above, but ensures that nothing in Dir is interpreted as a command
compound compound terms are tried to be expanded to an existing directory

The default options for libs are

load(Load=false)
index(Idx=true)
homonyms(Hmns=true)
type(Type=lib)

Command

One of

homonyms(From): attach homonyms From pack
init(Lib)
init(Lib, Cxt): declare initilization call (library can be loaded without this firing, if so needed, as is the case for lib_mkindex/1)
suggests(Lib)
suggests(Lib, SugOpts): it is likely you need Lib for full functionalilty. If Lib is a known library it is loaded other wise nothing is loaded.
This is useful for fringe functionalities that depend on external libraries, where we do not want the average user to do anything if library (Lib) is not there. See lib_suggests/2 for details of how to enable warning messages.
promise(Pred, Load): Pred is needed for functionality and it can be found by loading Load, but it will only happen at Pred's first call.
expects(Pid, Mess)
expects(Pid, Mess, Call): complains if Pid is not defined at loading time. Mess should be a debug style message with one ~w which will be called with Pid as its printing argument. If call is present, is called after the printing of the message.
version(Vers, Date): return version and publication date

Opts

index(Idx): whether to load indices
homonym(Hnym): whether to load homonym file-locators
load(Load): whether to load the main entry point of Repo
mode(Mode=self): makes missing message more acurate (other value: suggests)
suggest(Dn=true): suggest the library is downloaded if it is not locally installed ?
type(Type): enforce a particular type of repository (pack or lib)

The defaults depend on whether Repo is a pack or a lib.

Packs get defaults [load(true),index(true),homonym(false),type(pack)]
Libs get defaults [load(false),index(true),homonym(true),type(lib)]
When invoked with code attaching operands (SysLibrary, Pack or Lib) the predicate will first load anything that needs to be loaded in their native module and then import predicates from that module. Attaching a lib or pack means that the predicates pointed to by indices and by file name from the target pack/lib become available to the importee. Option index(Idx) controls whether LibIndex.pl based indices are attahced whereas homonym(Hmns) control the attachment of the file names from within the filesystem of the target.

For example to only import the interface predicates of pack ex1 use
```
?- lib(ex1, [type(pack),load(true),index(false),homonym(false)]).
```
Assume that ex1 is a pack that is not installed on your Prolog installation, but you have its sources unpacked on local dir /tmp/ex1/ you can load it interface predicates with:

For example to only import the interface predicates of pack ex1 use
```
?- lib('/tmp/ex1', [type(pack),load(true),index(false),homonym(false)]).
```
Assume there is a file src/lib/foo.pl in ex1 defining predicate foo/1, then you can load its code with
```
?- lib('/tmp/ex1', [type(pack),load(true),index(false),homonym(true)]).
?- lib(foo/1).
```
The above will first load foo.pl (by means of matching its filename to the predicate name) into ex1: and then assuming that this loaded foo/1 it will import it into current context (here this is =user+).

Assuming foo.pl also defines predicate bar/2 and there is a file src/LibIndex.pl within ex1 containing the line
```
lib_index( bar, 2, swipl(_), user, 'lib/foo.pl' ).
```
Then the code for foo_bar/2 can be loaded with
```
?- lib('/tmp/ex1', [type(pack),load(true),index(true),homonym(false)]).
?- lib(bar/2).
```
Pack lib can be used to create and access skeleton packs. These packs, may load very little interface code but their code base can be loaded on demand and piece-meal. That is if a specific non-interface predicate is required, it will be located and loaded along with all its dependencies.

An example of such a pack is stoics_lib. The following commands: 1. load the minimal interface,, 2, load the code for a specific non-interface predicate.
```
?- lib(stoics_lib).
?- lib(kv_decompose/3).
```
The above two directives can be shortened to:
```
?- lib(stoics_lib:kv_decompose/3).
```
```
?- lib( version(2:5:0, date(2019,5,8)) ).
true.
```
@author nicos angelopoulos @version 2:5 2019/5/8 @tbd when predicate is missing from stoics_lib while loading from b_real, we get clash between main and lazy, error should be clearer (the pred select_all/3 was actually not defined in file either)