Production MozCentral Unittest Moves – to the Build Network

So the last task in the move from QA network to Build network is moving the production mozilla-central unittest boxes.

This will take place on Friday September 19th, at 7am PDT

The new production buildbot is already up and running smoothly (currently reports to the UnitTest tinderbox tree) so there should hopefully be very little impact when this switch over happens.

Basically, the new unittest buildbot slaves will start reporting to the Firefox tinderbox tree, and the current slaves will stop reporting there.

This means you will be looking at new slave names. There are 2 buildslaves for each platform and their names are as follows:

Linux mozilla-central moz2-linux-slave07 dep unit test
Linux mozilla-central moz2-linux-slave08 dep unit test
MacOSX Darwin 9.2.2 moz2-darwin8-slave01 dep unit test
MacOSX Darwin 9.2.2 moz2-darwin8-slave02 dep unit test
WINNT 5.2 mozilla-central moz2-win32-slave07 dep unit test
WINNT 5.2 mozilla-central moz2-win32-slave08 dep unit test

Thanks to Jesse Ruderman and others for working so hard on bug #450637, as it means we are now running unit tests on win32 VMs with very consistent results.

As soon as this is done, and working well I can tie up the loose ends, close out some bugs, update the config files and start looking at some new projects…like the test results – and try server unittests.

Continuing saga of the 1.9 Unittest Move

When we left off, there was a check error happening across all Linux slaves and a reftest failure on the Win32 ones.

Update #1: A bug (450637) has been filed on that win32 failure, and also I brought the physical boxes back from sleep to be up on the new 1.9 master alongside their VM counterparts. We should know in the next hour or so if the reftest failure is consistent on all of them.

Update #2: The check error on Linux was due to the placement of a simple .sqlite file bug-365166.sqlite to be specific. This file was in /tmp and not in the slave build dir and thus, escaped during chown. Being owned by buildbot instead of cltbld was the cause of the access denied errors. Huge thanks to Cesar and Sdwilsh for looking at that test with me and for catching this anomaly. I’ve filed a bug (450665)to remove the offending placement so that this doesn’t happen again in the future. Files shouldn’t be getting created outside of the build dir, creates a whole mess of problems.

Speaking of mess:

Ew. That’s all I can say. I’ve been watching this waterfall obsessively (more than usual) as it has displayed a bruised variety of colours, mostly *Not* green.

In other news, something I noticed while upgrading the windows slaves:

Really? I didn’t know that people _chose_ IE. I thought it just came with the OS. I wish they would choose their words more carefully.

Back to the unittest trenches.

Update on the Unittest 1.9 move

In order to streamline the buildslave pool, the names of the following unittest 1.9 slaves were changed when we switched networks yesterday.

All of these machines now run Buildbot 0.7.7 and the latest Twisted & Python.

The Linux machines had their names changed and user changed – they are the same VMs as before:

qm-centos5-01 –> fx-linux-1.9-slave07
qm-centos5-02 –> fx-linux-1.9-slave08
qm-centos5-04 –> fx-linux-1.9-slave09

The Mac machines are the same ones as before, only a user change:

qm-xserve01 –> bm-xserve20
qm-xserve06 –> bm-xserve21

The two non-pgo windows machines are now VMs, the pgo box is the same VM that it was before – with a user change and a new 30GB fcal drive added for building on

qm-win2k3-01 –> fx-win32-1.9-slave07
qm-win2k3-02 –> fx-win32-1.9-slave08
qm-win2k3-pgo01 –> fx-win32-1.9-slave09

At the moment all three Linux boxes are experiencing errors in Check :

gmake[2]: Leaving directory `/builds/slave_new/trunk_centos5_8/mozilla/objdir/storage/build’
gmake[2]: Entering directory `/builds/slave_new/trunk_centos5_8/mozilla/objdir/storage/test’
../../_tests/xpcshell-simple/test_storage/unit/test_bug-365166.js: FAIL
../../_tests/xpcshell-simple/test_storage/unit/test_bug-365166.js.log:
>>>>>>>
*** Storage Tests: Trying to close!
*** Storage Tests: Trying to remove file!
*** test pending
[Exception… “Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [mozIStorageService.openDatabase]” nsresult: “0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)” location: “JS frame :: ../../_tests/xpcshell-simple/test_storage/unit/test_bug-365166.js :: test :: line 22” data: no]
*** FAIL ***

<<<<<<<
../../_tests/xpcshell-simple/test_storage/unit/test_bug-393952.js: PASS
../../_tests/xpcshell-simple/test_storage/unit/test_bug-444233.js: PASS

And all three Win32 boxes are having the same 1 test fail in Reftest:

REFTEST UNEXPECTED FAIL: file:///E:/slave/trunk_2k3_8/mozilla/layout/reftests/bugs/212563-1.html

Please contact me if you have any ideas about what could be causing these.

— Lukas