Version 39
Created by akostadinov on Aug 14, 2007 5:02 AM. Last modified by therfert on Jun 26, 2012 12:15 PM.

    This page is out dated.

    Use Eng Ops docs instead of this page: https://engineering.redhat.com/trac/eng-ops-mw-qe/wiki/MwEnvJenkins

    Overview

    We want to use NFS to simplify storage management. As well ssh and not JNLP slaves that is under question if it is a good idea. Plus some additional configuration so we can start things with a limited and not administrator account. And some scripts to remount NFS on every start and make a slave to run from the same location as on UNIX, load common configuration (MCAST_ADDR, MY_TESTIPs, java options, etc.) and that is. As long as we use ssh that can/should be passed to ssh for execution so we have svn control over it and don't need to modify all the slaves for a change.

     

    Used software

     

    We use cygwin with password auth due to limitations of windows to authorize users without a password. As well ssh server is the only means found for now that doesn't line buffer on the server side. So telnet and winexe are not yet suitable and telnet will probably never be.

     

    For the NFS client - MS SFU. It comes with nice unix utilities like kill. For Win 2003 R2 that should come bundled. As well we are supporting CIFS but not for workspace. As well WinR2k3 R2 slaves are only supported with local workspace because NFS locking doesn't work from privileged ports.

     

    PSkill we use to kill stale java processes so have it in path (install pstools).

     

    More Detailed Instructions

    Create Hudson user (see Win2k3 R2 setup)

    • Create a limited user account named hudson

    • Make it a member of the backup Operators group (dunno if required for the case but otherwise it could be unable to start in the background)

    • on win2k3 allow cmd.exe to be executed by the hudson user (again dunno if required any more... it was required for cruisecontrol started by task scheduler on system start-up)

    • Be sure to login at least once as hudson so home directory gets created!

    • Set the password for the account to never expire and like the other win slaves

    Install MS SFU

    • you can skip this and use the smb bridge as highly problematic
      • 2008 and vista
        • don't support easy mapping so anon access only possible
        • had problems running java off NFS but no problems from CIFS
      • xp and 2000 don't support AFAICT
      • 2003 R2 has problems with using priviledge ports for locking
    • see Win2k3 R2 setup below for more details after installation

    • Get installation binaries from qa_tools on the qa smb filer

    • install components - client and mapper + updates (see readme in the dir)

    • map users to unix uid numbers to match their home directories (probably only hudson user using passwd file)

    • use hard mount

    Install cygwin sshd

    See: Standard Cygwin installation in JBoss QE Lab

     

    Install pskill

    1. download pstools (probably can be found on NFS)

    2. unzip somewhere and set global PATH variable or put in system32

    3. IMPORTANT Run pskill once when logged in as hudson. Then accept license agreement. If you don't do that hudson startup script will hang.

     

    Start/stop scripts

    I tricked you ;p no stop script.

     

    The issue here is that we have to use ssh with password auth due to windows "security model". To be able to use password auth from script we have to trick ssh that it doesn't run from on a terminal and are using perl for the purpose. See the attached manually.txt for more information.

     

    The scripts are already under subversion control, so ignore attached files. The local startSlave.bat is not used any more. Everything is on the network storage.

     

    Procedure:

    1. use "/qa/home/hudson/launchWinSshSlave.sh host conf_file <feature list>" as startup command.

    2. launchWinSshSlave.sh executes sshWin.sh

    3. sshWin.sh accomplishes ssh connection using echo_pass.sh and setsid.pl then executes startSlave.bat

    4. the conf file in launchWinSlave/conf directory features a command for ssh to execute on the remote host.

    5. <feature list> is list of features - actually batch files located in launchWinSlave/init.d used to setup environment. For example see config for win2k3 and XP below:

      1. launchWinSshSlave.sh host1 nfs-sh allwin fixtemp javaloc mem1g nfslocat nfstools32 ips javaopts nfslaunch

      2. launchWinSshSlave.sh host2 cifs-sh allwin fixtemp javaloc mem700m nfslocat nfstools32 ips javaopts nfslaunch

     

    That creates a pretty flexible foundation for configuring new slaves needing just to write and reuse batch files in the init.d directory.

     

    Slave configuration:

     

    Besides startup command choose wisely working directory. It should be h:\hudson\hudson_workspace for current scripts on network storage aware windows slaves. Drive T is /qa/tools in case you need some utility (ant, java, etc) but these locations should provided in environment variables by the batch scripts executed.

     

    Advanced Start-Up (you don't need that any more - see above):

     

    You can't use quotes for the startup command so use as startup command "bash -c $ -- eval export MAX_JAVA_MEM=-Xmx700m; exec /qa/home/hudson/launchWinSlave.sh dev29;" to specify memory.

     

    Non-network storage slaves

    I use the same startup script as with network storage slaves. Differences:

    • Startup script begins with downloading slave.jar from the QA http server using wget from the cygwin installation

    • no mounting drives

    • use some local directory as a working directory

    • have utilities - java, ant, maven, svn, etc. locally

     

    Useful commands

    runas /user:mymachine\hudson cmd
runas /user:mymachine\cruisecontrol "explorer /separate"
runas /user:mymachine\cruisecontrol "C:\WINDOWS\system32\posix.exe /u /c /bin/ksh -l"

    MSAD and remote desktop with non-administrator users

    http://oreilly.com/pub/a/windows/2004/10/05/Remote_Desktop.html

    Basically:

    1. add the user to the "Remote Desktop Users" from the AD users and computers snap-in
    2. Add "Remote Desktop Users" to gpedit.msc  Local Security Policy or Default Domain Controller Policy: Computer Configuration\Windows Settings\Security Settings\Local Policies\User Rights Assignment and is called "Allow log on through Terminal Services."

    Links with old information

     

    Win2k3 R2 detailed setup

       * windows update
   * setup ntp time sync if not yet configured "net time /setsntp:clock.corp.redhat.com"
   * install windows components (you need setup CD2)
      * Subsystem for UNIX-based Applications
      * Othen Network File and Print Services
         * exclude server for NFS
         * include User Name Mapping
   * install "Utilities and SDK for UNIX-based Applications" as suggested by setup
      * custom installation
      * everything except visual studio debugger seems potentially useful to me
      * I set setuid in case that functionality happens to be needed (we don't care about security too much)
      * don't choose case sensitive settings
   * create a user named "hudson" in a group hudsongroup
      * set the standard hudson pass
      * make member of these groups
         * Remote Desktop Users
         * TelnetClients
   * Start regedit (that's needed to mount when conns are allowed from privileged ports only)
      * HKLM\Software\Microsoft\Client for NFS\CurrentVersion\Default
      * Add UseReservedPorts DWORD value "1"
      * Restart the Client for NFS service to allow the change to take effect.
   * Still could't figure out how to do the same for nfslock
   * setup NFS client and User Name mapping
      * administrative tools -> Microsoft Services For Network File System
         * Right click -> properties on Client for NFS
            * Hard mount
            * from file permissions allow group write
         * Right click -> properties on User Name Mapping
            * Use password and group files
            * use c:\etc\ passwd and group as files
            * from simple mapping select use simple maps
         * Fill in passwd and group files with data for:
           root,cruisecontrol,test_cc,hudson
           you can copy/paste unix entries
         * mapping
            * expand User Name Mapping
            * right click user maps and group maps to enable "show simple maps"
            * right click user maps to create map of root user with jbossqa and Administrator
            * right click group maps to map hudson with hudsongroup and root with Administrators
   * enable SUA telnet (skip that)
      * start ksh from SUA utilities
      * vi /etc/inetd.conf
      * reboot or kill -HUP <inetd pid>
      * will be used instead of cygwin ssh when hudson above 1.164 installed with text mode slave
   * install cygwin
      * see above
   * add the private IPs to the network interface
   * set computer name and DNS suffix

     

    Windows version specific notes

    Windows 2k

    This one is crap about mounts. Once you mount a drive in a session you can neither mount/use the same letter and network path in another session, nor umount the sessions. The only fix I've found is restart. So hudson start-up script should umount all shares with "net use * /delete /y".

    As well I'm still not having the 2k machine work properly but made it somehow work with textmode slave.

     

    Windows 2k3

    Once you mount a drive in one session, you can only mount the same letter in another session with the same network path. This holds for XP as well I think.