WineD3D Benchmarking Part 1: Auto-Building from Git

I’ve been following wine-git a lot recently, and especially now that they’re starting to put out wine-1.6 release candidates. I’m especially interested in seeing the evolution of the DirectX performance.

One of the Codeweavers developers, Stefan Dösigner, has been running regular performance benchmarks tracking D3D development with Phoronix-Test-Suite, and it’s great to see the changes over time. However, I have one major problem with his results: He’s using a much older hardware for his tests. They’re run on a 8-year old 2-core CPU (Athlon 64 X2) and a 7-year old GPU (Geforce 7600GT). What does that really tell me about Wine’s performance on newer hardware? I wanted to know for myself.

To start with, because I wanted to be able to build from wine-git instead of using the Ubuntu Wine PPA, as well as rebuilding with “-march=native” and using Clang instead of GCC. (Through my personal uses of  Wine with a few games, I’ve found performance better when it’s built with Clang, although it varies from game to game, and is sometimes worse. For the benchmarking, I intend to run everything as built by GCC, since the earlier versions needed patches to compile via Clang.)

Over a series of iterations, I’ve thrown together some bash scripts which do the building for me:

This accepts an optional command to run inside the chroot instead of launching an interactive shell. The purpose of the script itself is to prepare the bind-mounts (mount -o bind) and dpkg diversions in the chroot, and then remove them after exiting.

  • Bind mounts are needed to present the chroot with a working set of /proc, /sys, and /dev/pts, as well as to allow apt-in-the-chroot to share the package cache with the host’s /var/cache/apt/archives and not re-download or waste disk space for updates.
  • Dpkg diversions are needed  for /sbin/start-stop-daemon and /sbin/initctrl to avoid problems with package upgrades in the chroot. Many postinst scripts don’t like it when a call to start a service fails after an upgrade.

This uses dpkg-scanpackages, dpkg-scansources, apt-ftparchive and gpg to create a proper locally signed archive for use with apt-get.

This is the main beast. It accepts two parameters: a flag that tells it which patchset to use (vanilla, or full), and an optional git version tag or commit id:

  1. Update to the most recent Git, then reset to the specified commit (if given)
  2. Derive the version numbers to use: With the versioning scheme package-upstreamversion-localversion “git describe –abbrev=0” becomes the package-upstreamversion, and the difference between that and “git describe” becomes localversion, and we append “-1” on the end of that to create a distinct version. (The use of the git commit ID as part of the localversion is not kosher: however, this is only in my script. The actual package parses the commit ID as part of upstreamversion, and localversion becomes simply “-1”)

    • Commit 407584f: git describe –abbrev=0 produces “wine-1.6-rc2” and git describe produces “wine-1.6-rc2-32-g407584f” so we result in a full package version of “wine1.5-1.6-rc2-32-g407584f-1”
    • Commit bdc5c32: git describe –abbrev=0 produces “wine-1.5.29” and git describe produces “wine-1.5.29-99-gbdc5c32” so we result in a full package version of “wine1.5-1.5.29-32-gbdc5c32-1”
    • Git tag wine-1.5.7: git describe –abbrev=0 produces “wine-1.5.7” and git describe also produces “wine-1.5.7” so we result in a full package version of “wine1.5-1.5.7-1”
  3. Build .orig.tar.bz2 from git, now that we have the full upstream version number, and then extract into the temporary build directory.
  4. Find the appropriate debian.tar.gz and apply it to the build directory.
  5. Depending on the specified flag, look for a vanilla or full patchset. Check to see if there’s an existing debian.tar.gz that matches our upstream version. If not, use the basic “debian.tar.gz” The vanilla version is just that: no patches applied to the upstream source (except when required to get a proper compile, as was required with 1.5.30, which didn’t install all built components), and builds with GCC. The full patchset includes winepulse and whatever else is in the Ubuntu package, as well as any of my local changes, and builds withClang.
  6. Update changelog: the debian.tar.gz has a generic recent changelog entry where we need to replace the date and the version number (“###822DATE###” and “###VERSION###” are used as placeholders)
  7. Enter chroot and build i386 version: This is usually where things fail, especially when using the full patchset.
  8. If that succeeds, build the amd64 version
  9. Move to local archive and run

Using ccache, gcc-4.8.1, and building the 34 releases from 1.5.0 through 1.6-rc2, it took a little over 7.5 hours to compile (Phenom II X6). With the vanilla patchset (i.e. no patches to upstream), the whole run built with no errors, which was a wonderful surprise to wake up to.

The next thing will be running the actual benchmarks. This will take considerably longer as each benchmark run takes at least 90 minutes (about 2.5 days total runtime for all wine versions), but at least I’ve already worked out the framework and scripts to do that as well. That’s part 2.

Tags: , , ,

One Comment

  • […] After Part 1, I have waited to run any benchmarks, until it looked like they were getting close to actually releasing Wine 1.6. I wanted to run them all at once, so that the results weren’t skewed by system updates between runs (Another problem I have with looking at Stefan’s historic results: I don’t know which improvements are due to wine development, and which were from video driver/kernel/library updates). […]