MirOS Manual: cvs-distfiles(7)

CVS-DISTFILES(7)             BSD Reference Manual             CVS-DISTFILES(7)

NAME

     cvs-distfiles - fetch distfiles for MirPorts via CVS

SYNOPSIS

     CVS_DISTREPO=:ext:cvs.example.com:/cvs
     CVS_DISTDATE=2007-09-22
     CVS_DISTTAGS=1_0_release
     CVS_DISTMODS=src

     SVN_DISTPATH=https://svn.example.com/projects/foo/trunk
     SVN_DISTFILE=foo-svn
     SVN_DISTREV=123
     SVN_DISTDIR=foo

DESCRIPTION

     This document describes how the "fetch distfile via CVS" (or Subversion)
     method in MirPorts operates.

The problem

     Often, software is not distributed as nicely as we would like to have it
     - in a versioned tarball, gzip'd or compress'd. Sometimes, tarballs are
     not versioned, broken, or just too old. But almost universally, source
     code is available via a revision control system - such as CVS or Subver-
     sion.

     This manual documents a method to fetch sources via CVS, while retaining
     control over correct checksums etc.

From a user's point of view: CVS

     To enable the _CVS_DISTF fetch method, you have to put a few liness into
     your port Makefile. As usual, learning by an example is "the way to go" -
     prominent examples are the three ports essentials/pkgtools,
     games/bsdgames, and devel/libtool.

     The user interface consists of a few variables you can set:

     CVS_DISTREPO  the value of the -d option to cvs

     CVS_DISTDATE  (optional) value of -D option to cvs checkout

     CVS_DISTTAGS  (optional) value of -r option to cvs checkout

     CVS_DISTMODS  module(s) to check out, separated by space

     CVS_DISTFILE  (optional) file name prefix

     Note that if CVS_DISTREPO does not start with a colon, CVSREADONLYFS is
     set automatically, which is not compatible with the OpenBSD version of
     anoncvssh.

     Note: while both CVS_DISTDATE and CVS_DISTTAGS are optional, it is re-
     quired to set one of them because HEAD checkouts are a moving target and
     thus not supported benefitting checksumming.

     Setting CVS_DISTREPO${_i} enables _CVS_DISTF operation on a specific
     distfile. ${_i} can be empty or a single decimal digit, in case you need
     more than one distfile.

     Once operation has started, _CVS_DISTF${_i} is set to the name of the
     file created in ${FULLDISTDIR} (/usr/ports/Distfiles), in the following
     scheme: basename[-dateext][-tagext].mcz

     The basename is CVS_DISTFILE${_i}, or (if unset) CVS_DISTMODS${_i} with
     dashes removed. The dateext (only if CVS_DISTDATE is set) is exactly
     this, but with all non-numerical parts removed. The tagext (if
     CVS_DISTTAGS) is the tag, with all characters except digits, letters,
     dash or underscore removed, and a capital T prepended. This ensures in
     almost all cases that _CVS_DISTF is unique.

     Then, _CVS_FETCH${_i} is set to a command to "retrieve the file" if it is
     not yet there. If any _CVS_DISTF${_i} is set, archivers/mpczar is pulled
     in as a fetch dependency automatically.

     All _CVS_DISTF${_i} are added to _DISTFILES automatically, so there is no
     need to do this. Now you only need to set a PKGNAME (unless the default
     is fine to you) and you are done.

From a user's point of view: Subversion

     To enable the _CVS_DISTF fetch method, you have to put a few liness into
     your port Makefile. As usual, learning by an example is "the way to go" -
     prominent example is the port sysutils/vesautils.

     The user interface consists of a few variables you can set:

     SVN_DISTDIR   Name of the directory to check out into (top-level package
                   dir). Defaults to the basename of SVN_DISTREPO.

     SVN_DISTFILE  Optional file name prefix. Defaults to the basename of
                   SVN_DISTDIR.

     SVN_DISTREPO  Full path to the repository and subdirectory to check out.

     SVN_DISTREV   Revision number to check out.

     Setting SVN_DISTREPO${_i} enables _CVS_DISTF operation on a specific
     distfile. ${_i} can be empty or a single decimal digit, in case you need
     more than one distfile.

     Once operation has started, _CVS_DISTF${_i} is set to the name of the
     file created in ${FULLDISTDIR}  (/usr/ports/Distfiles), in the following
     scheme: basename-revext.mcz

     The basename is SVN_DISTFILE${_i}. The revext is the letter 'r' followed
     by the numerical SVN_DISTREV revision. This ensures in almost all cases
     that _CVS_DISTF is unique.

     Then, _CVS_FETCH${_i} is set to a command to "retrieve the file" if it is
     not yet there. If any _CVS_DISTF${_i} is set, archivers/mpczar and
     devel/subversion are pulled in as fetch dependencies automatically.

     All _CVS_DISTF${_i} are added to _DISTFILES automatically, so there is no
     need to do this. Now you only need to set a PKGNAME (unless the default
     is fine to you) and you are done.

More interesting possibilities

     The infrastructure automatically defaults to setting the (first) distri-
     bution directory:
           WRKDIST?=${WRKDIR}/${CVS_DISTMODS}
           WRKDIST?=${WRKDIR}/${SVN_DISTDIR}

     If CVS_DISTMODS (without ${_i}) contains more than one module, you must,
     consequentially, set WRKDIST yourself (also if you differ from what the
     infrastructure thinks is a sensible default).

     The package name is a requirement which still must be set manually, un-
     less the new default of ${_CVS_DISTF:R}-${DASH_VER} (CVS) or
     ${SVN_DISTFILE}-${SVN_DISTREV}-${DASH_VER} (SVN) suits your port, which
     will cover 95% of the cases. FOR CVS, THIS ONLY WORKS IF YOU USE
     CVS_DISTDATE *AND* *NOT* CVS_DISTTAGS. Another example is to just set a
     sensible one yourself, like libtool does (which pulls from the branch-1-5
     tag):
           PKGNAME=libtool-1.5.23a.${CVS_DISTDATE:S!/!!g}-${DASH_VER}

     Of course, this needs more maintenance.

What is working behind the curtains

     The infrastructure defines a target ${FULLDISTDIR}/${_CVS_DISTF${_i}} for
     each file, which is invoked by 'mmake fetch' automatically and only if
     the file does not exist. However, the target itself also checks for the
     existence of the distfile, because in some rare cases all dependencies
     are invoked.

     First thing is to execute a 'mmake fetch-depends' so that mpczar is only
     built if one of the _CVS_DISTF files is missing, and subversion is only
     built if needed as well. This is done already inside the check for ex-
     istence.

     After that, _CVS_FETCH${_i} is invoked on the file and the size is
     checked (cloned from a "normal" fetch process).

     Checksumming, extracting and so on are processed with no difference; the
     .mcz file extension yielded an addition to EXTRACT_CASES, of course.

     Operation of _CVS_FETCH:

     The script /usr/ports/infrastructure/scripts/mkmcz is invoked with a few
     parameters (target, repo, date, tag, modules) and then creates said *.mcz
     file, as follows:

     Prepare a temporary directory (mkdtemp), change into it, invoke 'cvs
     checkout', or 'svn checkout', call mpczar on the current directory with
     an appropriate set of "ignore files" ('CVS' and '.svn').

     mpczar(1) then archives all files (no directories, pipes, devices, sym-
     bolic links, and other funky stuff) in ASCII sort order into an System V
     CPIO archive with CRC, using the new -M option to cpio(1), which tells it
     to "normalise" the archive: serialise inode numbers, zero out mtime, own-
     ership and device information, and, for efficiency, store hard links'
     file contents only once. This is safe in the scenario (it would not be
     safe if one of the files to pack had a hard link to outside the area
     which is packed, because its link count is increased by one then). The
     CPIO archive is LZ compressed using the 'mpczar.z' helper (really just
     compress(1) in disguise), which is a mathematical function.

     This achieves that the same unordered set of input files always creates
     the same archive (with the same checksum).

SEE ALSO

     bsd.port.mk(5), ports(7), mpczar(1).

MirOS BSD #10-current          January 7, 2014                               2

Generated on 2014-07-04 21:17:45 by $MirOS: src/scripts/roff2htm,v 1.79 2014/02/10 00:36:11 tg Exp $

These manual pages and other documentation are copyrighted by their respective writers; their source is available at our CVSweb, AnonCVS, and other mirrors. The rest is Copyright © 2002‒2014 The MirOS Project, Germany.
This product includes material provided by Thorsten Glaser.

This manual page’s HTML representation is supposed to be valid XHTML/1.1; if not, please send a bug report – diffs preferred.