Copyright © 2004-2015 Matthew Seaman. All rights reserved.
portindex, portdepends, cache-init, cache-update, find-updated and make-readmes are a set of perl scripts built around the common core of the FreeBSD::Portindex perl modules. Their use is to generate and maintain the ports INDEX or INDEX-N files speedily and efficiently (where N is the FreeBSD major version number). Ultimately they work in a very similar way to the standard make index command, except that the FreeBSD::Portindex tools keep a cache of the make describe output from each port and timestamps for Makefiles and pkg-descr files, and can update that cached data incrementally as the ports tree itself is updated.
Incremental updating is much faster than building the INDEX entirely from scratch every time. Once the cache has been initialised, all of the changes made over the course of a day or a week can be incorporated in a few minutes, and an up to date INDEX file produced in a few seconds. Being able to quickly generate a fresh INDEX file from the cache is also exceedingly handy.
Why would you build your own version of the INDEX rather than just using make fetchindex to download a recently generated default version or taking the INDEX file produced by portsnap? Firstly: because your locally built INDEX will reflect the configuration choices you've made either through the OPTIONS dialogues that many ports use, or by editing /etc/make.conf. Secondly: because portindex gives you an easy way to incorporate information about local ports into your INDEX.
The cache-update process will now detect modifications to any files read during the generation of the INDEX and update the cached data for the affected ports. No other input is required, although you can still update the cache the old way; by parsing the output of running svn or the output of portsnap update. All of these methods serve to work out which ports have been modified or deleted recently — this includes ports that have been affected indirectly by a change to any of the ports Makefiles involved in building them, and so need their cached description updated.
find-updated has been radically rewritten. Rather than running find over the ports tree, now it simply compares timestamps in exactly the same way as cache-update. It's really just a read-only preview of what cache-update would do.
The portdepends script prints out the dependency tree for a port: recent discussion on the freebsd-ports mailing list suggested that there weren't many good tools available to do this. FreeBSD::Portindex already had 90% of the code required, so adding this was a no-brainer.
Vizualizing dependency graphs is not easily done from the original textual output of portdepends, so with version 2.5, portdepends has acquired the ability to print out the dependency graph in GraphViz dot generic input format, allowing pictorial representations of dependency trees to be easily generated. Here is the a fairly simple example: the dependency graph for portindex itself.
make-readmes does exactly what it says on the tin. It makes a tree of README.html files — one for each port or category and one at the top of the tree. The result should be pretty much equivalent to running make readme in $PORTSDIR (the generated html in particular is about as bad as the original.) Advantages of the make-readmes script are speed and that it can create the tree of README.html files in an arbitrary directory, not just based at $PORTSDIR.
Please see the POD documentation for each of the commands for details. However, routine usage will typically be something like this:
Prerequisites: you will need to have installed a recent version of Perl, preferably perl-5.16.2 (lang/perl5.16) or later. Additionally you will need p5-BerkeleyDB-0.51 (databases/p5-BerkeleyDB) and one of the Berkeley Database Library ports such as db5-5.3.28 (databases/db5). Other Perl or BDB versions will work, but have not been thoroughly tested.
Optionally you can also install graphviz-2.28.0_1 (graphics/graphviz), but this is only required to render the portdepends output into an image format; not for portdepends itself to run.
Of course, installing via the ports-mgmt/p5-FreeBSD-Portindex port is the recommended method.
However, for completeness' sake, here are the instructions for how to do a manual installation. Download and unpack the portindex tar-ball, and install:
% tar -xvf FreeBSD-Portindex-N.NN.tar.xz
% cd FreeBSD-Portindex-N.NN
% perl Makefile.PL
% make
% make test
# make install
% rehash
This is a slow step (slower than make index) but it only needs to be done occasionally.
# mkdir /var/db/portindex
# chown user:group /var/db/portindex
% cache-init
That creates the portindex cache file (about 55MiB currently) containing all of the make describe output, and timestamp data for all the various Makefiles and pkg-descr files required to generate the INDEX-N entry for the port. It also contains a timestamp file showing the last time that the cache was updated. Nb. while you can run any of the FreeBSD::Portindex programs as root safely enough, on general principles I recommend using a non-privileged user account instead.
Then, for day to day use:
Before generating a new INDEX-N file, update the FreeBSD::Portindex cache with the details of any recently updated ports. The recommended way to do this is by comparing the timestamps recorded in the cache with the current state of what is on disk.
% cache-update
The old methods of parsing the output of csup or portsnap update, and the various other alternatives still work. See the POD pages for cache-update and find-updated for more information.
Repeat as necessary if you update the ports tree several times between generating new INDEX-N files.
# portindex -o /usr/ports/INDEX-N
Steps (iii) and (iv) together should take a minute or so — more if one of the commonly included Makefiles like /usr/ports/Mk/bsd.gnome.mk has been modified. Even so, it will be very much faster than building the INDEX-N file from scratch each time you update your ports.
The latest version is FreeBSD::Portindex-3.6 released on 18th October 2015.
| Version | Size | Date | Comments |
|---|---|---|---|
| 3.6 | 56224 | 2015/10/18 | Don't require $(PORTSDIR) to be explicitly included in DEPENDS lines. Move to GitHub. |
| 3.5 | 56572 | 2015/08/23 | Fix perl 5.20 warning. Remove references to the obsolete $OPTIONSFILE and use $OPTIONS_FILE exclusively. |
| 3.4 | 56396 | 2014/05/04 | Account for the differences between bmake(1) in FreeBSD 10.0 and the original fmake(1). |
| 3.3 | 56192 | 2013/12/26 | Drop all support for old style OPTIONS, account for variant locations where options may be stored. |
| 3.2 | 56036 | 2013/03/31 | Really fix behaviour where /usr/ports is a sym-link. |
| 3.1 | 55876 | 2013/03/30 | Drop cvsup-output and cvsup-checkouts input formats, add svn-up. Fix behaviour where /usr/ports is a sym-link. |
| 3.0 | 57096 | 2012/12/02 | Detect ports that use options correctly with OPTIONSng. Switch to pragma use parent for inheritance |
| 2.9 | 56728 | 2012/06/01 | Adjust Endemic and Ubiquitous Makefile settings to account for OPTIONSng |
| 2.8 | 56240 | 2012/02/25 | Radical redisgn of how the cache operates |
| 2.7 | 52612 | 2011/11/07 | Bugfixes, documentation improvements, fix dependency tracking (again!) |
| 2.6 | 51828 | 2011/10/26 | Fix bug that could crash portindex. |
| 2.5 | 51700 | 2011/10/07 | GraphViz mode for portdepends. |
| 2.4 | 50724 | 2011/08/28 | 1) portdepends short output mode. 2) Bugfix handling LIB_DEPENDS and RUN_DEPENDS. |
| 2.3 | 50095 | 2011/03/19 | New portdepends script, update EndemicMakefiles setting. |
| 2.2 | 46188 | 2009/08/09 | Object inheritance reorganisation, add alternate SHLIBS index format output for portindex. |
| 2.1 | 32939 | 2008/04/07 | Extend the plain mode to read portsnap output. |
| 2.0 | 32206 | 2007/08/05 | OPTIONS processing, strict mode, reworked internal data structures. |
| 1.9 | 29297 | 2007/02/04 | DEPENDS variable and supporting code removed from bsd.ports.mk |
| 1.8 | 29342 | 2006/09/17 | Fix crash when handling broken dependency information. |
| 1.7 | 29110 | 2006/07/23 | Improved handling of Master/Slave relations between ports. Don't quit on errors. |
| 1.6 | 27889 | 2006/06/25 | Handle Makefile.local additions to ports. Improve handing of port category changes. |
| 1.5 | 24514 | 2006/01/29 | Fix handling of empty port categories when running cache-init. |
| 1.4 | 24390 | 2005/02/19 | Fix recursion loop, enable processing cvsup-cvs mode output. |
| 1.3 | 24030 | 2005/01/21 | Fix programming mistake that prevented cache-update's cvsup-checkouts mode from working. |
| 1.2 | 23975 | 2005/01/16 | Respect $PORTSDIR. Make some Makefile lists configurable. |
| 1.1 | 21958 | 2004/11/05 | Add processing based on included makefiles. |
| 1.0 | 19366 | 2004/11/01 | First public release |
| 0.3 | 18602 | 2004/10/30 | BETA - Reorganise class hierarchy, add find-updated |
| 0.2 | 15899 | 2004/10/23 | BETA - Bug fixes |
| 0.1 | 15850 | 2004/10/22 | BETA - First publicly available version |
Also available: SHA256 Checksums for all available release versions
Please report any bugs to me, Matthew Seaman with as much detail as possible so I can try and reproduce them. Please try and include:
FreeBSD version — uname -a output.
Perl version — perl -V output.
Transcript of session showing how the problem occurs.
As well as anything else you think is relevant.
Now that I have moved the development repository onto GitHub you can also use GitHub's issue tracker or, even better, send me a pull request.