Jenkins, cygpath plugin does not convert paths

Question

I'm using Jenkins with Cygpath plugin installed.

I have some scripts that is working in Cygwin bash without problems

#!/bin/bash

pwd

SCRIPTFILENAME=$(readlink -f $0)
SCRIPTPATH=`dirname $SCRIPTFILENAME`
cd $SCRIPTPATH
(python check_for_clip_duplicates.py) || exit $?

But when i'm trying to build it with Jenkins i have en error:

$ D:\cygwin\bin\cygpath -w d:\cygwin\bin\bash.exe
[default] $ D:\cygwin\bin\bash.exe -xe C:\Windows\TEMP\hudson2178008588278192726.sh
cygwin warning:
MS-DOS style path detected: C:\Windows\TEMP\hudson2178008588278192726.sh
  Preferred POSIX equivalent is: /cygdrive/c/Windows/TEMP/hudson2178008588278192726.sh
  CYGWIN environment variable option "nodosfilewarning" turns off this warning.
  Consult the user's guide for more details about POSIX paths:
http://cygwin.com/cygwin-ug-net/using.html#using-pathnames
+ sh tools/generate_all.sh pc compress reset --use-texture-packer
tools/generate_all.sh: line 2: $'\r': command not found
tools/generate_all.sh: line 3: $'pwd\r': command not found
tools/generate_all.sh: line 4: $'\r': command not found
tools/generate_all.sh: line 7: cd: /cygdrive/d/Jenkins/workspace/default/tools

: No such file or directory
C:\Python27\python.exe: can't open file 'check_for_clip_duplicates.py': [Errno 2] No such     file or directory
tools/generate_all.sh: line 8: exit: 2
: numeric argument required
Build step 'Execute shell' marked build as failure
Finished: FAILURE

In Jenkins configuration in Shell executable i have: d:\cygwin\bin\bash.exe.

Main problem is that cygdrive paths does not work:

tools/generate_all.sh: line 7: cd: /cygdrive/d/Jenkins/workspace/default/tools

: No such file or directory

Any ideas?

Answer 1

The script is most likely failing because your cd $SCRIPTPATH is failing (due to trailing \r characters) and it then tries to find check_for_clip_duplicates.py in the current directory of the script (instead of tools directory), which isn't there.

Why do all previous commands fail and only the last one executes? Cause the last one doesn't actually have a newline, therefore no \r there.

Download Notepad++, it's an essential tool and has great support for file EOL conversion (your problem with \r)

• Open Notepad++
• Click File -> New
• Note at the bottom right in the status bar, it will show Dos\Windows (default EOL)
• Click Edit -> EOL Conversion -> UNIX Format
• Note at the bottom right in the status bar, it will now show UNIX
• Please type out (don't copy paste) the content of your generate_all.sh script
• Save this file to your tools/generate_all.sh location. You may not notice the difference, but the line endings are invisible in normal view.
• Important: If you are using SVN or other SCM to checkout into Jenkins' workspace, you need to save your changes back to SVN/SCM!

Finally, to avoid this happening again, you can put the dos2unix command into your Execute Shell build step that you use to trigger generate_all.sh:

dos2unix toos/generate_all.sh
tools/generate_all.sh pc compress reset --use-texture-packer

Pro tip:
To actually view line endings, in Notepad++, click View -> Show Symbol -> Show All Characters

EDIT:
You call python from your script, which in turn calls Windows executable C:\Python27\python.exe. As an external executable, it has no awareness of any Jenkins plugins (cygpath) or even Jenkins itself, for that matter.

Analyzing your question further, there is little need for Cygwin in the first place, you can do all that with regular Execute Windows Batch command build step

cd is Windows equivalent of pwd.
$WORKSPACE is an environment variable pointing to the path of your job's workspace, and you know the script is in tools folder.
The || exit $? can be substituted with || exit %ERRORLEVEL% but it is completely redundant if this is the last line in the build step (script, actually). Jenkins will use the exit code of the last statement of the build step as the exit code for the whole build step

So change your whole cygwin build step to regular Execute Window Batch build step with the following:

cd
cd $WORKSPACE/tools
C:\Python27\python.exe check_for_clip_duplicates.py 

If your python directory is already in the %PATH% variable, you can simply use python instead of full path.

Answer 2

Looks like you have some Windows line endings in your file - try running dos2unix on it:

$ dos2unix /cygdrive/c/Windows/TEMP/hudson2178008588278192726.sh

Then try running the script again.