Merging of Git Repositories

If you ever need to merge multiple git repositories into a single repository and also keep the history, then i have just the script for you. This script will keep your branches and tags, but since tags often are called the same, the tags and branches of a repository is renamed so that the original repository name is prefixed.

If you had the following branches and tags in your project “myproject”:

  • master
  • develop
  • feature/my-feature
  • 0.1.0
  • 0.1.1

Then they would become:

  • myproject/master
  • myproject/develop
  • myproject/feature/my-feature
  • myproject/0.1.0
  • myproject/0.1.1

Further if the original repository has more than one file and/or directory in the root, then these get moved in to a sub directory so that we don’t have conflicts with other projects.

To use the project simply create a file with the list of repository URLs you want to merge, e.g.:

# Merge repositories:
git@github.com:4treesCH/strolch-wc-inspector.git
git@github.com:4treesCH/strolch-wc-auth.git
git@github.com:4treesCH/strolch-wc-localize-behavior.git
git@github.com:4treesCH/strolchjs.git
git@github.com:4treesCH/strolch-wc-styles.git
git@github.com:4treesCH/strolch.git

Then you can perform the merge using the following script as so:

./mergeGitRepositories.sh test repos.list

So here is the script:

https://paste.ubuntu.com/p/BhKvmMnqFQ/

Maven & Git: Releases and Hotfixes

When you write an application you want a simple way to release a new major version, but also to release bugfixes (here i call them hotfixes) to a released version. If you write Java applications, chances are, you are using Maven and then as a source control you might be using GIT. Maven offers a nice way to set versions, and Git makes it easy to create releases and tags.

I personally use the git branch model described by Vincent Griessen. There development is done on the develop branch, and when a release is done, then a release branch is created. This branch is called release/x.x.x. There the version is updated and a tag is created from this release. If you then create a hotfix, you do that on this release branch, create the new tag, and then merge those changes back down to the develop branch (and perhaps any feature branches).

I have created to scripts which help you to easily create these releases and hotfixes.

release.sh: This is the first script to do a release. It’s input is the source branch, mostly develop, and the version to created. It will create a new release/x.x.x branch, create a temp branch to create the tag, create the tag, cleanup and then push to remote:

#!/bin/bash

# usage
if [ $# != 2 ] ; then
 echo -e "Usage: ${0} <source_branch> <version>"
 exit 1
fi

# Confirm
sourceBranch=${1}
releaseVersion=${2}
echo -e "INFO: Do you want to release version ${releaseVersion} from branch ${sourceBranch}? y/n"
read a
if [[ "${a}" != "y" && "${a}" != "Y" ]] ; then
 exit 0;
fi

# validate tag and release branch do not exist
echo -e "INFO: Fetching tags and branches and validating they do not exist..."
if ! git fetch --all --tags > /dev/null ; then
 echo -e "ERROR: Tags and branches could not be fetched!"
 exit 1
fi
if git tag | grep "${releaseVersion}" > /dev/null ; then
 echo -e "ERROR: Tag already exists!"
 exit 1
fi
if git branch --all | grep "release/${releaseVersion}" > /dev/null ; then
 echo -e "ERROR: Tag already exists!"
 exit 1
fi

# validate tag does not exist
if git tag | grep "${releaseVersion}" > /dev/null ; then
 echo -e "ERROR: Tag already exists!"
 exit 1
fi

# make sure gpg-agent is available and loaded
echo -e "\nINFO: Searching for gpg-agent..."
if ! which gpg-agent ; then
 echo -e "ERROR: gpg-agent missing!"
fi
if ! gpg-agent 2>&1 | grep "running and available" ; then
 echo -e "WARN: gpg-agent not running, trying to start..."
 if ! gpg-agent --daemon ; then
 echo -e "ERROR: Failed to initialize gpg-agent, please make sure it is up and running before continuing!"
 exit 1
 fi
 if ! gpg-agent 2>&1 | grep "running and available" ; then
 echo -e "ERROR: Failed to initialize gpg-agent, please make sure it is up and running before continuing!"
 exit 1
 fi
fi

# Checkout source branch
echo -e "\nINFO: Checking out source branch ${sourceBranch}"
if ! git checkout ${sourceBranch} ; then
 echo -e "ERROR: Failed to checkout branch ${sourceBranch}"
 exit 1
fi

# create new release branch, and temp tag
echo -e "\nINFO: Creating release and temp branches..."
if ! git checkout -b release/${releaseVersion} ; then 
 echo -e "ERROR: Failed to create release branch!"
 exit 1
fi
if ! git checkout -b temp ; then
 echo -e "ERROR: Failed to create temp branch!"
 exit 1
fi

# set release version
echo -e "\nINFO: Setting version..."
if ! mvn versions:set -DgenerateBackupPoms=false -DnewVersion=${releaseVersion} > /dev/null ; then
 echo -e "ERROR: Failed to set new version!"
 exit 1
fi

# build
echo -e "\nINFO: Doing a build with new version..."
if ! mvn clean package -DskipTests > /dev/null ; then
 echo -e "ERROR: Failed to build with new version!"
 exit 1
fi

# commit to tag
echo -e "\nINFO: Creating tag..."
if ! git add . ; then
 echo -e "ERROR: Failed to git add"
 exit 1
fi
if ! git commit -m "[Project] Set new version ${releaseVersion}" ; then
 echo -e "ERROR: Failed to git commit"
 exit 1
fi
if ! git tag --sign --message "[Project] New Version ${releaseVersion}" ${releaseVersion} ; then
 echo -e "ERROR: Failed to git tag"
 exit 1
fi

# cleanup
echo -e "\nINFO: Cleaning up..."
if ! git checkout release/${releaseVersion} ; then
 echo -e "ERROR: Failed to checkout release branch"
 exit 1
fi
if ! git branch -D temp ; then
 echo -e "ERROR: Failed to delete temp branch"
 exit 1
fi

# git push
echo -e "\nINFO: Release ${releaseVersion} created. Do you want to push to origin? y/n"
read a
if [[ "${a}" == "y" || "${a}" == "Y" ]] ; then
 echo -e "INFO: Pushing to origin..."
 if ! git push origin release/${releaseVersion} ; then
 echo -e "ERROR: Failed to push release branch"
 exit 1
 fi
 if ! git push origin ${releaseVersion} ; then
 echo -e "ERROR: Failed to push tag"
 exit 1
 fi
 echo -e "\nINFO: Pushed release branch and tag for release version ${releaseVersion}"
else
 echo -e "WARN: Release not pushed!"
fi

echo -e "\nINFO: Release ${releaseVersion} created."
echo -e "INFO: Don't forget to update snapshot version!"

exit 0

hotfix.sh: This is the second script to do a hotfix to a release. It’s input is the release version, i.e. 1.0.0 or 1.1.0, and the hotfix version to created i.e. 1.0.1 or 1.1.1. It will check out the release branch i.e. release/1.0.0, create a temp branch to create the tag, create the tag, cleanup and then push to remote:

#!/bin/bash

# usage
if [ $# != 2 ] ; then
 echo -e "Usage: ${0} <release_version> <hotfix_version>"
 exit 1
fi

# Confirm
releaseVersion="${1}"
releaseBranch="release/${1}"
hotfixVersion="${2}"
echo -e "INFO: Do you want to make hotfix version ${hotfixVersion} from release branch ${releaseBranch}? y/n"
read a
if [[ "${a}" != "y" && "${a}" != "Y" ]] ; then
 exit 0;
fi

# validate tag does not exist
echo -e "INFO: Fetching tags and branches and validating they do not exist..."
if ! git fetch --all --tags > /dev/null ; then
 echo -e "ERROR: Tags and branches could not be fetched!"
 exit 1
fi
if git tag | grep ${hotfixVersion} > /dev/null ; then
 echo -e "ERROR: Tag already exists!"
 exit 1
fi

# make sure gpg-agent is available and loaded
echo -e "\nINFO: Searching for gpg-agent..."
if ! which gpg-agent ; then
 echo -e "ERROR: gpg-agent missing!"
fi
if ! gpg-agent 2>&1 | grep "running and available" ; then
 echo -e "WARN: gpg-agent not running, trying to start..."
 if ! gpg-agent --daemon ; then
 echo -e "ERROR: Failed to initialize gpg-agent, please make sure it is up and running before continuing!"
 exit 1
 fi
 if ! gpg-agent 2>&1 | grep "running and available" ; then
 echo -e "ERROR: Failed to initialize gpg-agent, please make sure it is up and running before continuing!"
 exit 1
 fi
fi

# Checkout release branch
echo -e "\nINFO: Checking out release branch ${releaseBranch}"
if ! git checkout ${releaseBranch} ; then
 echo -e "ERROR: Failed to checkout branch ${releaseBranch}"
 exit 1
fi

# create temp branch
echo -e "\nINFO: Creating temp branch..."
if ! git checkout -b temp ; then
 echo -e "ERROR: Failed to create temp branch!"
 exit 1
fi

# set hotfix version
echo -e "\nINFO: Setting version..."
if ! mvn versions:set -DgenerateBackupPoms=false -DnewVersion=${hotfixVersion} > /dev/null ; then
 echo -e "ERROR: Failed to set new version!"
 exit 1
fi

# build
echo -e "\nINFO: Doing a build with new version..."
if ! mvn clean package -DskipTests > /dev/null ; then
 echo -e "ERROR: Failed to build with new version!"
 exit 1
fi

# commit to tag
echo -e "\nINFO: Creating tag..."
if ! git add . ; then
 echo -e "ERROR: Failed to git add"
 exit 1
fi
if ! git commit -m "[Project] Set new version ${hotfixVersion}" ; then
 echo -e "ERROR: Failed to git commit"
 exit 1
fi
if ! git tag --sign --message "[Project] New Version ${hotfixVersion}" ${hotfixVersion} ; then
 echo -e "ERROR: Failed to git tag"
 exit 1
fi

# cleanup
echo -e "\nINFO: Cleaning up..."
if ! git checkout ${releaseBranch} ; then
 echo -e "ERROR: Failed to checkout release branch"
 exit 1
fi
if ! git branch -D temp ; then
 echo -e "ERROR: Failed to delete temp branch"
 exit 1
fi

# git push
echo -e "\nINFO: Hotfix ${hotfixVersion} created. Do you want to push to origin? y/n"
read a
if [[ "${a}" == "y" || "${a}" == "Y" ]] ; then
 echo -e "INFO: Pushing to origin..."
 if ! git push origin ${hotfixVersion} ; then
 echo -e "ERROR: Failed to push tag"
 exit 1
 fi
 echo -e "\nINFO: Pushed hotfix tag ${hotfixVersion}"
else
 echo -e "WARN: Release not pushed!"
fi

echo -e "\nINFO: Hotfix ${hotfixVersion} created."

exit 0

Have fun using the scripts, and don’t hesitate to give me comments!

How do you merge two git repositories?

I’ve had the issue of needing to merge multiple git repositories into one repository with preserving the history, branches and tags. This isn’t a trivial feat, as the repositories have a large history, many branches and tags and of course names of those branches and tags overlap.

While trying to solve this issue i searched the net for solutions, often landing on a stackoverflow.com page of some kind. Gathering this information i have managed to put together my own script, which seems to solves the issue rather well.

#!/bin/bash
#
################################################################################
## Script to merge multiple git repositories into a new repository
## - The new repository will contain a folder for every merged repository
## - The script adds remotes for every project and then merges in every branch
##  and tag. These are renamed to have the origin project name as a prefix
##
## Usage: mergeGitRepositories.sh <new_project> <my_repo_urls.lst>
## - where <new_project> is the name of the new project to create
## - and <my_repo_urls.lst> is a file contaning the URLs to the respositories
## which are to be merged on separate lines.
##
## Author: Robert von Burg
## eitch@eitchnet.ch
##
## Version: 0.2.0
## Created: 2015-06-17
##
################################################################################
#

# disallow using undefined variables
shopt -s -o nounset

# Script variables
declare SCRIPT_NAME="${0##*/}"
declare SCRIPT_DIR="$(cd ${0%/*} ; pwd)"
declare ROOT_DIR="$PWD"


# Detect proper usage
if [ "$#" -ne "2" ] ; then
 echo -e "ERROR: Usage: $0 <new_project> <my_repo_urls.lst>"
 exit 1
fi


# Script functions
function failed() {
 echo -e "ERROR: Merging of projects failed:"
 echo -e "$1"
 exit 1
}

function commit_merge() {
 current_branch="$(git symbolic-ref HEAD 2>/dev/null)"
 CHANGES=$(git status | grep "working directory clean")
 MERGING=$(git status | grep "merging")
 if [[ "$CHANGES" != "" ]] && [[ "$MERGING" == "" ]] ; then
 echo -e "INFO: No commit required."
 else
 echo -e "INFO: Committing ${sub_project}..."
 if ! git commit --quiet -m "[Project] Merged branch '$1' of ${sub_project}" ; then
 failed "Failed to commit merge of branch '$1' of ${sub_project} into ${current_branch}"
 fi
 fi
}


## Script variables
PROJECT_NAME="${1}"
PROJECT_PATH="${ROOT_DIR}/${PROJECT_NAME}"
REPO_FILE="${2}"
REPO_URL_FILE="${ROOT_DIR}/${REPO_FILE}"


# Make sure the REPO_URL_FILE exists
if [ ! -e "${REPO_URL_FILE}" ] ; then
 echo -e "ERROR: Repo file ${REPO_URL_FILE} does not exist!"
 exit 1
fi


# Make sure the required directories don't exist
if [ -e "${PROJECT_PATH}" ] ; then
 echo -e "ERROR: Project ${PROJECT_NAME} already exists!"
 exit 1
fi


# create the new project
echo -e "INFO: Creating new git repository ${PROJECT_NAME}..."
echo -e "===================================================="
cd ${ROOT_DIR}
mkdir ${PROJECT_NAME}
cd ${PROJECT_NAME}
git init
echo "Initial Commit" > initial_commit
# Since this is a new repository we need to have at least one commit
# thus were we create temporary file, but we delete it again.
# Deleting it guarantees we don't have conflicts later when merging
git add initial_commit
git commit --quiet -m "[Project] Initial Master Repo Commit"
git rm --quiet initial_commit
git commit --quiet -m "[Project] Initial Master Repo Commit"
echo


# Merge all projects into th branches of this project
echo -e "INFO: Merging projects into new repository..."
echo -e "===================================================="
for url in $(cat ${REPO_URL_FILE}) ; do

 # extract the name of this project
 export sub_project=${url##*/}
 sub_project=${sub_project%*.git}

 echo -e "INFO: Project ${sub_project}"
 echo -e "----------------------------------------------------"

 # Fetch the project
 echo -e "INFO: Fetching ${sub_project}..."
 git remote add "${sub_project}" "${url}"
 if ! git fetch --no-tags --quiet ${sub_project} 2>/dev/null ; then
 failed "Failed to fetch project ${sub_project}"
 fi

 # add remote branches
 echo -e "INFO: Creating local branches for ${sub_project}..."
 while read branch ; do 
 branch_ref=$(echo $branch | tr " " "\t" | cut -f 1)
 branch_name=$(echo $branch | tr " " "\t" | cut -f 2 | cut -d / -f 3-)

 echo -e "INFO: Creating branch ${branch_name}..."

 # create and checkout new merge branch off of master
 git checkout --quiet -b "${sub_project}/${branch_name}" master
 git reset --hard --quiet
 git clean -d --force --quiet

 # Merge the project
 echo -e "INFO: Merging ${sub_project}..."
 if ! git merge --quiet --no-commit "remotes/${sub_project}/${branch_name}" 2>/dev/null ; then
 failed "Failed to merge branch 'remotes/${sub_project}/${branch_name}' from ${sub_project}"
 fi

 # And now see if we need to commit (maybe there was a merge)
 commit_merge "${sub_project}/${branch_name}"

 # relocate projects files into own directory
 if [ "$(ls)" == "${sub_project}" ] ; then
 echo -e "WARN: Not moving files in branch ${branch_name} of ${sub_project} as already only one root level."
 else
 echo -e "INFO: Moving files in branch ${branch_name} of ${sub_project} so we have a single directory..."
 mkdir ${sub_project}
 for f in $(ls -a) ; do
 if [[ "$f" == "${sub_project}" ]] || 
 [[ "$f" == "." ]] || 
 [[ "$f" == ".." ]] ; then 
 continue
 fi
 git mv -k "$f" "${sub_project}/"
 done

 # commit the moving
 if ! git commit --quiet -m "[Project] Move ${sub_project} files into sub directory" ; then
 failed "Failed to commit moving of ${sub_project} files into sub directory"
 fi
 fi
 echo
 done < <(git ls-remote --heads ${sub_project})


 # checkout master of sub probject
 if ! git checkout "${sub_project}/master" 2>/dev/null ; then
 failed "sub_project ${sub_project} is missing master branch!"
 fi

 # copy remote tags
 echo -e "INFO: Copying tags for ${sub_project}..."
 while read tag ; do 
 tag_ref=$(echo $tag | tr " " "\t" | cut -f 1)
 tag_name=$(echo $tag | tr " " "\t" | cut -f 2 | cut -d / -f 3)

 # hack for broken tag names where they are like 1.2.0^{} instead of just 1.2.0
 tag_name="${tag_name%%^*}"

 tag_new_name="${sub_project}/${tag_name}"
 echo -e "INFO: Copying tag ${tag_name} to ${tag_new_name} for ref ${tag_ref}..."
 if ! git tag "${tag_new_name}" "${tag_ref}" 2>/dev/null ; then
 echo -e "WARN: Could not copy tag ${tag_name} to ${tag_new_name} for ref ${tag_ref}"
 fi
 done < <(git ls-remote --tags ${sub_project})

 # Remove the remote to the old project
 echo -e "INFO: Removing remote ${sub_project}..."
 git remote rm ${sub_project}

 echo
done


# Now merge all project master branches into new master
git checkout --quiet master
echo -e "INFO: Merging projects master branches into new repository..."
echo -e "===================================================="
for url in $(cat ${REPO_URL_FILE}) ; do

 # extract the name of this project
 export sub_project=${url##*/}
 sub_project=${sub_project%*.git}

 echo -e "INFO: Merging ${sub_project}..."
 if ! git merge --quiet --no-commit "${sub_project}/master" 2>/dev/null ; then
 failed "Failed to merge branch ${sub_project}/master into master"
 fi

 # And now see if we need to commit (maybe there was a merge)
 commit_merge "${sub_project}/master"

 echo
done


# Done
cd ${ROOT_DIR}
echo -e "INFO: Done."
echo

exit 0

Java Equals and HashCode by Reflection

Writing proper ”equals()” and ”hashCode()” methods
A class which is to be used in Maps and Sets must always have their equals() and hashCode() method overridden so that they reflect the data structure of the actual object.

This means that the fields of the object which are required for equality must be taken into account when overriding these methods. The JavaDoc of Object.equals() and Object.hashCode() go into further detail, but it remains to be mentioned that if a field has a unique identifier then this will mostly suffice for the equality implementation.

When using the Eclipse IDE, implementing the equals() and hashCode() method are a matter of:

  • Opening the context menu in the class’s source editor
  • Opening the Source menu
  • Selecting Generate hashCode() and equals()…
  • Select on the fields required for uniqueness
  • Further configure the dialog and then selecting ok

Note: For further information, see Joshua Bloch’s book Effective Java which describes this issue further.

Performance issues
Apache Commons has two classes HashCodeBuilder and EqualsBuilder as an easy way of implementing these methods. These classes use reflection to dynamically build the hashCode or check equality of two instances.

They should be avoided as they can have a serious impact on performance as the following test will show.

Test Class:

  public static class TestClass {
    public static final int GENERATED = 0;
    public static final int REFLECTION = 1;
    public static final int COMMONS_BUILDER = 2;
    public static int hashCodeBuilding = GENERATED;
    private int a;
    private int b;
    private String c;
    private String d;

    public TestClass(final int a, final int b, final String c, final String d) {
      this.a = a;
      this.b = b;
      this.c = c;
      this.d = d;
    }

    @Override
    public int hashCode() {
      if (hashCodeBuilding == REFLECTION) {
        return HashCodeBuilder.reflectionHashCode(this);
      }
      if (hashCodeBuilding == COMMONS_BUILDER) {
        return new HashCodeBuilder()
                      .append(this.a)
                      .append(this.b)
                      .append(this.c)
                      .append(this.d)
                      .toHashCode();
      }

      final int prime = 31;
      int result = 1;
      result = prime * result + this.a;
      result = prime * result + this.b;
      result = prime * result + ((this.c == null) ? 0 : this.c.hashCode());
      result = prime * result + ((this.d == null) ? 0 : this.d.hashCode());
      return result;
    }

    @Override
    public boolean equals(final Object obj) {

      if (hashCodeBuilding == REFLECTION) {
        return EqualsBuilder.reflectionEquals(this, obj);
      }

      if (this == obj) {
        return true;
      }
      if (obj == null) {
        return false;
      }
      if (getClass() != obj.getClass()) {
        return false;
      }

      TestClass other = (TestClass) obj;
      if (hashCodeBuilding == COMMONS_BUILDER) {
        return new EqualsBuilder().append(this.a, other.a)
                     .append(this.b, other.b)
                     .append(this.c, other.c)
                     .append(this.d, other.d)
                     .isEquals();
      }

      if (this.a != other.a) {
        return false;
      }

      if (this.b != other.b) {
        return false;
      }

      if (this.c == null) {
        if (other.c != null) {
          return false;
        }
      }
      else if (!this.c.equals(other.c)) {
        return false;
      }

      if (this.d == null) {
        if (other.d != null) {
          return false;
        }
      }
      else if (!this.d.equals(other.d)) {
        return false;
      }

      return true;
    }
  }

JUnit Test method:

  @Test
  public void hashCodePerformanceTest() {
    TestClass a = new TestClass(22, 11, "bla bla", "no no");
    TestClass b = new TestClass(52, 63, "dfgdfgdfg", "xv3dfg1");

    System.out.println("First pass...");
    runTest(a, b, TestClass.REFLECTION);
    runTest(a, b, TestClass.COMMONS_BUILDER);
    runTest(a, b, TestClass.GENERATED);

    System.out.println("Second pass...");
    runTest(a, b, TestClass.REFLECTION);
    runTest(a, b, TestClass.COMMONS_BUILDER);
    runTest(a, b, TestClass.GENERATED);

    System.out.println("Third pass...");
    runTest(a, b, TestClass.REFLECTION);
    runTest(a, b, TestClass.COMMONS_BUILDER);
    runTest(a, b, TestClass.GENERATED);

    System.out.println("Fourth pass...");
    runTest(a, b, TestClass.REFLECTION);
    runTest(a, b, TestClass.COMMONS_BUILDER);
    runTest(a, b, TestClass.GENERATED);

    System.out.println("Fifth pass...");
    runTest(a, b, TestClass.REFLECTION);
    runTest(a, b, TestClass.COMMONS_BUILDER);
    runTest(a, b, TestClass.GENERATED);
  }

  private void runTest(final TestClass a, final TestClass b, final int hashCodeBuilding) {
    TestClass.hashCodeBuilding = hashCodeBuilding;
    long start = System.nanoTime();
    for (int i = 0; i < 10000; i++) {
      a.equals(b);
      a.hashCode();
    }

    long time = System.nanoTime() - start;
    if (hashCodeBuilding == TestClass.REFLECTION) {
      String msg = String.format("With reflection took %.3f ms", (time / 1000000.0D));
      System.out.println(msg);
    }
    else if (hashCodeBuilding == TestClass.COMMONS_BUILDER) {
      String msg = String.format("With apache commons builder took %.3f ms", (time / 1000000.0D));
      System.out.println(msg);
    }
    else {
      String msg = String.format("With eclipse generated took %.3f ms", (time / 1000000.0D));
      System.out.println(msg);
    }
  }

Test result:

First pass...
With reflection took 72.796 ms
With apache commons builder took 1.689 ms
With eclipse generated took 0.607 ms
Second pass...
With reflection took 48.158 ms
With apache commons builder took 1.199 ms
With eclipse generated took 0.151 ms
Third pass...
With reflection took 48.240 ms
With apache commons builder took 1.095 ms
With eclipse generated took 0.151 ms
Fourth pass...
With reflection took 47.274 ms
With apache commons builder took 2.264 ms
With eclipse generated took 0.150 ms
Fifth pass...
With reflection took 46.475 ms
With apache commons builder took 1.199 ms
With eclipse generated took 0.160 ms

Postfix Satellite configuration: Remote SMTP with auth

If you have ever run into the problem that your computer will not send e-mails directly, then this might help.

Install Postfix and configure it as a satellite server. This was done on a Ubuntu desktop installation, so I don’t know what Fedora and co. will say.

In the /etc/postfix/main.cf file add or modify the following lines:

relayhost = smtp.gateway.ch
smtp_sasl_auth_enable = yes
smtpd_tls_auth_only = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous

This configures postfix to relay all e-mails over the defined host. Now the sasl_passwd file must be created with the credentials for the SMTP gateway:

smtp.gateway.ch user@gateway.ch:password

Now create the map of the password file and reload postfix:

sudo postmap /etc/postfix/sasl_passwd
sudo /etc/init.d/postfix reload

Test the configuration by sending a test e-mail:

echo -e "From: someone@somewhere.ch\nTo: some@otherplace.com\nSubject: Hello World\n\Body: \nHello beautiful world" | sendmail -t

And see what postfix said:

tail /var/log/mail.info

Have fun!

Makefile to compile C code

During my phases where I learn to write in C, I don’t want to keep creating new Makefiles. Thus after searching all over the net I managed to put together this simple Makefile. It expects your source code to be in ./src and it will put the compiled executables in ./bin

#
# Makefile to compile arbitrary C sources to an executable with the same name
# The source files are in the src/ sub directory
# The compiled executables are in the bin/ sub directory
#

# $@ is automatic variable that holds name of target
# $< is automatic variable that holds the name of the prerequisite

# defines the sub directory in which the sources are found
SRC_PATH=./src/
# defines the sub directory in which the executables are compiled to
BIN_PATH=./bin/

# tells make to search in $(SRC_PATH) for the .c files
VPATH = src:$(SRC_PATH)

# defines the name of the executables which are to be compiled
EXECUTABLES= $(patsubst $(SRC_PATH)%.c,%,$(shell ls $(SRC_PATH)*.c))

# default target which prepares the bin/ sub directory, and initiates the compilation
all: prepare $(EXECUTABLES)

# the target which is expanded to compile each found C source file
$(EXECUTABLES): %: %.c
	gcc $< -lm -o $(BIN_PATH)$@

# declare phony targets
.PHONY: prepare clean

# create the $(BIN_PATH) sub directory
prepare:
	mkdir -p $(BIN_PATH)
# remote the $(BIN_PATH) subdirectory
clean:
	rm -rf $(BIN_PATH)

Adding new LaTeX packages and styles

This is a quick explanation on how to install new LaTeX styles so that you can use them, but without having to change your system.

  • First download a style from CTAN
  • Then unpack the zip somewhere in your home folder
  • Copy the styles/packages to ~/texmf/tex
  • Run the command mktexlsr or texhash
  • Use the package and it should work

This helped me: StackOverflow

Ubuntu Netboot

Notes on getting Ubuntu netboot to work so that I don’t have to burn any CD’s

This document expects a DHCP server to be available in the network

Install tftp

sudo aptitude install tftpd-hpa tftp-hpa

Get netboot files:

cd /var/lib/tftpboot
sudo wget -r -nH -np --cut-dirs=8 http://archive.ubuntu.com/\
ubuntu/dists/maverick/main/installer-amd64/\
current/images/netboot
find ./ -name "index.*" | sudo xargs rm

Add the following to the DHCP subnet configuration:

next-server 10.0.0.68;
filename "pxelinux.0";

Now normal netboot should work.

If one wants to have liveCD, then carry on by installing NFS-Server:

sudo aptitude install nfs-kernel-server

Add export for the livecd in /etc/exports:

/var/lib/tftpboot/maverick 10.0.0.0/255.255.255.0(async,no_root_squash,no_subtree_check,ro)

Mount and copy liveCD contents to exported NFS share:

sudo mount -o loop /tmp/maverick-desktop-amd64.iso /mnt
sudo cp -ar /mnt/* /var/lib/tftpboot/maverick/

Add a new menu entry in ubuntu-installer/amd64/boot-screens/rqtxt.cfg

label maverickLiveCD
menu label ^Live CD
kernel maverick/casper/vmlinuz
append vga=788 boot=casper netboot=nfs \
nfsroot=10.0.0.68/var/lib/tftpboot/maverick initrd=maverick/casper/initrd.lz --

Restart NFS:

sudo /etc/init.d/nfs-kernel-server reload

If problems arise, try using the following boot append line:

append file=/var/lib/tftpboot/maverick/preseed/ubuntu.seed vga=788 \
boot=casper root=/dev/nfs netboot=nfs \
nfsroot=10.0.0.68:/var/lib/tftpboot/maverick initrd=maverick/casper/initrd.lz --

VirtualBox resolution problems in a headless environment

In certain cases running a VirtualBox VM in a headless linux environment can lead to a Windows XP having an undesired resolution.

I fixed this using the following two commands:

Global command:
VBoxManage setextradata global GUI/MaxGuestResolution 1920,1200

For a running VM:
VBoxManage controlvm "remote" setvideomodehint 1920 1200 24

I found the resolution in this forum:
VirtualBox Forums

aptitude search vs apt-cache search

Over the years I have grown used to the fact that when I search for a package on a debian based distro, then I use:
apt-cache search foo bar
The result was a list of all the packages which had the text ‘foo’ and ‘bar’ in the description.

Lately it has been discouraged to use apt-get and apt-cache, but one should rather use aptitude as this tool is more powerful.

It is more powerful, but it does not behave the same. So the following command:
aptitude search foo bar
would return any result where either ‘foo’ or ‘bar’ was in the package name, not the description.

Since I have gotten used to apt-cache way this is how I do it with aptitude:
aptitude search "~dfoo bar"

But! This command does not check package names. Package names are with ~n. Maybe someone out there can explain how I can combine the two correctly?