Back

FreeBSD::Portindex

Introduction

portindex, cache-init, cache-update and find-updated are a set of perl scripts built around the common core of the FreeBSD::Portindex perl modules. Their use is to generate and maintain the ports INDEX or INDEX-N files speedily and efficiently (where N is one of 5, 6, 7 or 8 depending on FreeBSD major version number). Ultimately they work in a very similar way to the standard make index command, except that the FreeBSD::Portindex tools keep a cache of the make describe output from each port, and can update that cached data incrementally as the ports tree itself is updated.

Incremental updating is much faster than building the INDEX entirely from scratch every time. Once the cache has been initialised, all of the changes made over the course of a day or a week can be incorporated in a few minutes, and an up to date INDEX file produced in a few seconds. Being able to quickly generate a fresh INDEX file from the cache is also exceedingly handy now that the INDEX files have been removed from CVS.

Why would you build your own version of the INDEX rather than just using make fetchindex to download a recently generated default version or taking the INDEX file produced by portsnap? Firstly: because your locally built INDEX will reflect the configuration choices you've made either through the OPTIONS dialogues that many ports use, or by editing /etc/make.conf. Secondly: because portindex gives you an easy way to incorporate information about local ports into your INDEX.

The cache-update process can either take the output of running cvsup over the ports-all collection, or it can read cvsup's checkouts file (usually /usr/sup/ports-all/checkouts.cvs:. in FreeBSD 4.x or /var/db/sup/ports-all/checkouts.cvs:. in 5.x or above) or it can read the output of portsnap update to work out which ports have been modified or deleted recently — this includes ports that have been affected indirectly by a change to any of the ports Makefiles involved in building them, and so need their cached description updated. Of course, it can also just read a simple list of the port origins to re-check, which you can generate using find-updated or how you will.

Installation and Operation — Quick Start

Please see the POD documentation for each of the commands for details. However, routine usage will typically be something like this:

0) Dependencies

Prerequisites: you will need to have installed a recent version of Perl, preferably perl-5.8.9_3 (lang/perl5.8) or perl-5.10.0_4 (lang/perl5.10). Additionally you will need p5-BerkeleyDB-0.39 (databases/p5-BerkeleyDB) and one of the Berkeley Database Library ports such as db46-4.6.21.4 (databases/db46). Other Perl or BDB versions probably will work, but have not been thoroughly tested.

i) Installation

Of course, installing via the ports-mgmt/p5-FreeBSD-Portindex port is the recommended method.

However, for completeness' sake, here are the instructions for how to do a manual installation. Download and unpack the portindex tar-ball, and install:

        % tar -xvf FreeBSD-Portindex-N.NN.tar.bz2
        % cd FreeBSD-Portindex-N.NN
        % perl Makefile.PL
        % make
        % make test
        # make install
        % rehash
    

ii) Initialise the Cache

This is a slow step (slower than make index) but it only needs to be done occasionally.

        # mkdir /var/db/portindex
        # chown user:group /var/db/portindex
        % cache-init
    

That creates the portindex cache file (about 27MiB currently) containing all of the make describe output, as well as details of all ports Makefiles required to build the port and whether the port is a slave port of some other one. It also contains a timestamp file showing the last time that the cache was updated. Nb. while you can run any of the FreeBSD::Portindex programs as root safely enough, on general principles I recommend using a non-privileged user account instead.

Then, for day to day use:

iii) Update the Cache

Before generating a new INDEX-N file, update the FreeBSD::Portindex cache with the details of any recently updated ports. The recommended way to do this is by parsing the output of cvsup or portsnap update, but there are several other alternatives. See the POD pages for cache-update and find-updated for other possibilities.

cvsup users should run:

       # script /tmp/cvsup.out \
            cvsup -g -L2 -h cvsupN.XX.freebsd.org \
            /usr/share/examples/cvsup/ports-supfile
       % cache-update -i /tmp/cvsup.out
    

portsnap users should run:

       # portsnap fetch
       # script /tmp/portsnap.out portsnap update
       % cache-update -f plain -i /tmp/portsnap.out
    

Repeat as necessary if you update the ports tree several times between generating new INDEX-N files.

iv) Generate the INDEX-N file

       # portindex -o /usr/ports/INDEX-N
    

Steps (iii) and (iv) together should take a minute or so — more if one of the commonly included Makefiles like /usr/ports/Mk/bsd.gnome.mk has been modified. Even so, it will be very much faster than building the INDEX-N file from scratch each time you update your ports.

Downloads

The latest version is FreeBSD::Portindex-2.2 released on 9^th August 2009.

Available versions

Also available: MD5 Checksums and SHA256 Checksums for all available release versions

Bugs

Please report any bugs to me, Matthew Seaman with as much detail as possible so I can try and reproduce them. Please try and include:

FreeBSD version — uname -a output.

Perl version — perl -V output.

Transcript of session showing how the problem occurs.

As well as anything else you think is relevant.

---

Copyright © 2004-2008 Matthew Seaman. All rights reserved.

Matthew Seaman

$Revision: 1.23 $ Last Modified $Date: 2009/08/09 16:13:59 $