Because it always takes me too much time to find out how to do it, I thought it was a good idea to make mental notes on my blog.
First, download p4merge from the perforce website and install it.
Add the path to p4merge to the git config:
git config --global --add difftool.p4merge.path "C:\Program Files\Perforce\p4merge.exe"
Then set p4merge to be the default diff tool:
git config --global --add diff.tool p4merge
While at it, let's also register p4merge as the default merge tool:
git config --global --add merge.tool p4merge