Development: BuildbotOnWindowsXp

From WxWiki

Jump to: navigation, search

chowette - 2008-09-14 - This is a work in progress, It is now more like a scratch pad to remember what I perform than a perfect guide.

Contents

List of softwares to install:

  • a working C++ compiler
  • SVN
  • python
  • buildbot who depends on
    • twisted who also need
      • pywin32
      • pycrypto
      • pyOpenSSL and OpenSSL
  • cppunit
  • stunnel

Install steps

Your compiler, I choose VC2008 to start with.

Visual C++ 2008 express

Download the VS2008ExpressWithSP1ENUX1504728.iso, burn the iso or mount it using deamon-tools. Run the setup for the Visual C++ studio. Important: Do not forget to run the IDE once so it can setup some registry keys.

Visual C++ 6

Use your favorite CD to install, but be sure to install the last service pack 6 as well. It will save you some internal compiler errors.

SVN

Download Setup-Subversion-1.7.0.msi to get the svn.exe command line tool.

You can also use tortoise to get a shell integration.

python 2.7

Get python-2.7.2.msi

setuptools

setuptools-0.6c8.win32-py2.5.exe

twisted

Twisted-11.1.0.win32-py2.7.exe I have run tests with

   x:>python c:\python27\scripts\trial.py twisted   

and got some errors :

   FAILED (skips=1565, expectedFailures=12, errors=2, successes=5208) in 1st try

pywin32

pywin32-216.win32-py2.7.exe

pycrypto

The installed version is 2.4.1

   easy_install pycrypto 

pyOpenSSL

The installed version is 0.13

   easy_install pyopenssl 

run twisted test again:

   FAILED (skips=1397, expectedFailures=12, errors=2, successes=5369)

OpenSSL

Win32OpenSSL_Light-1_0_0e.exe

   FAILED (skips=1397, expectedFailures=12, errors=2, successes=5369)

buildbot 0.8.5

unzip buildbot-slave-0.8.5.zip and run:

   x:\buildbot-slave-0.8.5\>python setup.py install

Go in a temporary folder and run the tests:

   x:\test>python c:\Python27\Scripts\trial.py buildslave.test

for now, I get some errors:

   FAILED (skips=9, errors=4, successes=85)

so I installed mock

mock 0.8.0beta4

   easy_install mock

And rerun the builslave test and go a

   PASSED (skips=9, successes=139)

cppunit

cppunit-1.12.1.tar.gz Unarchive, then open the file k:\VC9libs\cppunit-1.12.1\src\CppUnitLibraries.dsw. Answer "Yes to all" when visual studio ask for converting VC6 projects. Go to menu "build > batch build ..."; select every configuration for the projects cppunit, cppunit_dll and DllPlugInTester (10 targets).

stunnel

Get from it http://www.stunnel.org/download/binaries.html. Run the installer. Choose a path, then edit the stunnel.conf file

mine is:

; stunnel.conf
; Use it for client mode
client = yes
CAfile = bb.cert.pem
; display a pretty name in service list
; service =  wxWidget Buildbot tunnel ; for some reason, this line makes an error now !?! 

;cert = stunnel.pem
;debug = 7

[buildbot]
accept = 127.0.0.1:9989
connect = buildbot.tt-solutions.com:9990

So any connection to 127.0.0.1:9989 will be forwarded to buildbot.tt-solutions.com:9990 using a SSL tunnel

Start the tunnel

>stunnel.exe

Check if the tunnel is running and listening

>netstat -a 

You should see a line like:

TCP   <computerName>:9989   <computerName>:0    LISTENING*

Register stunnel as a service so it will start automatically

stunnel -install

Configure your slave

Use wxWidgets\build\buildbot\config\example.xml as a starting point. At that point, my config file is:

Checking result

The $(wx)/build/buildbot/tools/ folder contains a check.sh script to check configuration. I have used the following check.bat file to perform that test in my windows box:

@rem Naive convertion of check.sh
@rem Put me in $(wx)/build/buildbot/tools/ folder
@rem  usage : >check ..\config\example.xml
@rem This script assume that  xsltproc.exe and xmllint.exe are in %PATH%
@rem You can found the tool at  ftp://xmlsoft.org/libxml2/win32/


set INPUT=%1

mkdir test

echo generating %INPUT%
xsltproc --xinclude -o test/xslt.xsl embedded.xsl %INPUT%
echo Preprocessing %INPUT%
xsltproc --xinclude -o test/prep.xml test\xslt.xsl %INPUT%
echo Validating %INPUT%
xmllint --format --schema bot.xsd test\prep.xml > test\res.xml

making a sandbox

Maybe the best option is to run the build slave in a virtual computer so you can restore it in a second.

create an user

For now, I just created a user:

Go to the management console,

> mmc.exe /s C:\WINDOWS\system32\compmgmt.msc

Under 'user and local host\group', create a group called 'buildSlave'. Under 'user and local host\user', create a user called 'buildSlaveVC9'. Set it member of the 'buildSlave' group.

Into the explorer, right click on your C: drive and go to the security tab. Click on 'Add' to add the 'buildSlave' group. Select the group and deny writing right Use the same procedure to remove all rights to the group 'buildSlave' on every drive. Use the same procedure to add all rights to the user 'buildSlaveVC9' on buildslave path.

sandbox file

   @echo off
   set LOGDIR=K:\buildbot\logs\
   rem get date
   For /f "tokens=1-4 delims=/ " %%a in ('date /t') do (set date=%%c-%%b-%%a)
   rem get time
   FOR /f "tokens=*" %%G IN ('time/t') DO set _time=%%G
   SET _time=%_time:~0,2%h%_time:~3,2%
   
   rem log to a file
   echo %_time% %CD%# %* >> %LOGDIR%%date%.sandbox.log
   if /I %1 == VC9 call "C:\Program Files\Microsoft Visual Studio 9.0\VC\vcvarsall.bat" x86
   if /I %1 == VC9 set INCLUDE=%INCLUDE%;k:\buildbot\VC9libs\cppunit-1.12.1\include
   if /I %1 == VC9 set LIB=%LIB%;k:\buildbot\VC9libs\cppunit-1.12.1\lib
   
   if /I %1 == VC6 call "C:\Program Files\Microsoft Visual Studio\VC98\Bin\vcvars32.bat"
   if /I %1 == VC6 set INCLUDE=%INCLUDE%;k:\buildbot\VC6libs\cppunit-1.12.1\include
   if /I %1 == VC6 set LIB=%LIB%;k:\buildbot\VC6libs\cppunit-1.12.1\lib
   
   rem we need a new temp folder each day because nmake is so dumb and create names that collides once in a while
   set TEMP=k:\buildbot\slaves\XP VC9\temp\%date%
   set TMP=k:\buildbot\slaves\XP VC9\temp\%date%
   if exist %TEMP% goto afterTemp
   set lastPath=%CD%
   cd k:\buildbot\slaves\XP VC9\temp\
   mkdir %date%
   cd %lastPath%
   :afterTemp
  
   rem execute command
   %COMSPEC% /c %2 %3 %4 %5 %6 %7 %8 %9

lauch using different user

The runas command should work, but there is not way to set a password automatically, so i reverted to use pstool.

Download psTools from sysinternal on Microsoft http://www.microsoft.com/technet/sysinternals/utilities/psexec.mspx

 psexec -u buildSlaveVC9 -p <password for buildSlaveVC9> <the-command>

Avoid drwatson in unhandled exception of test

When a test fail to run with a unhandled exception, a window ask to send error informations, blocking the running of tests.

installing Windbg

First, download Windbg. Install it to the default location. (mine is C:\Program Files\Debugging Tools for Windows (x86)\)

run a terminal and type the following in one line

"C:\Program Files\Debugging Tools for Windows (x86)\cdb.exe" 
            -p %ld -e %ld -g -logo err.%Id.dbg.err -lines -c "!analyze -v;.ecxr;!for_each_frame dv /t;q"

It will install cdb as the default debugger, attaching to the process and executing the following:

  • -p %ld : the process id
  • -e %ld : the point of error
  • -g : avoid the initial breakpoint
  • -logo err.%Id.dbg.err : log everything to the file err.00000.dbg.err
  • -lines : use source code when available
  • -c : execute the following command in the debugger:
    • !analyze -v : verbose analysis the exception
    • ecxr : jump into the context of the exception
    • !for_each_frame dv /t : for each frame, display variables.
    • q : quit the debugger

installing windows symbols files

In order to have the name of windows function, download and install the Windows Symbol Packages (warning! huge file: 1Go after install for WinXpSP2)

After install, you need to add the path to environment variables so windbg can found them

_NT_SYMBOL_PATH=c:\WINDOWS\


TODO: add the current wxwidget lib path to the symbol path. Somthing like this should work.

  (for /d %x in (..\lib\*_dll) do @_NT_SYMBOL_PATH=%x;%_NT_SYMBOL_PATH%)
Personal tools