osx - A bash/wget script runs from command line but fails to run under launchd - output files not written -

i having terrible time getting bash script uses wget run launchd job. script designed automate downloading of daily new york times crossword puzzle. note need subscription (username , password) make work.

the original script (which lives @ http://web.mit.edu/lizdenys/public/crossword) works great , impressive , educational example of bash scripting (for me @ least). here orignal. modified use including user , pass , commenting out steps download pdf version (i pencil , paper).

-=-=-=-=-=-=begin bash script-=-=-=-=-=-=

#!/bin/bash  # crossword, # wget-based nytimes crossword downloader  # liz a. denys (liz@lizdenys.com) # last updated on may 13, 2015  # script downloads today's new york times daily crossword. # use, must change email , password information below # corresponds premium new york times account.  # current date. puzdate=`date "+%y-%m-%d"` pdfdate=`date "+%b%d%y"`  # login page. wget --no-check-certificate https://myaccount.nytimes.com/auth/login \   -o login.html &>/dev/null  # scrape token , expires values wget can auth. token=`grep token login.html | sed -e 's/^.*value="\([0-9a-f]\+\)".*$/\1/'` expires=`grep expires login.html | sed -e 's/^.*value="\([0-9a-f]\+\)".*$/\1/'`  # log in password. note: not work without replacing # username , password information. wget --post-data \   "userid=username%40domain.com&password=passwordvalue&is_continue=false&remember=true&token=$token&expires=$expires" \   --save-cookies=cookies.txt --keep-session-cookies --no-check-certificate \   -o /dev/null https://myaccount.nytimes.com/auth/login &>/dev/null  # download puzzle in .pdf , .puz formats. wget --load-cookies=cookies.txt \  http://www.nytimes.com/svc/crosswords/v2/puzzle/print/$pdfdate.pdf \   &>/dev/null wget --load-cookies=cookies.txt \   http://www.nytimes.com/svc/crosswords/v2/puzzle/daily-$puzdate.puz \   &>/dev/null  # clean workspace. rm cookies.txt rm login.html 

-=-=-=-=-=-=end bash script-=-=-=-=-=-=

i made required edit , script ran command line on first attempt!

since don't have internet access every day , since (as far know) puzzles no longer available day after appear, decided automate job run every day. i've used cron (for incremental backups on sparcstation) no problems ever (using make select files write tape - reveals vintage), no, apple wants me use launchd. i'm happy learn new , useful task learn enough launchd , plists perform extremely simple job. not so!!

here 1 of many plists failed:

-=-=-=-=-=-=begin .plist file-=-=-=-=-=-=

<?xml version="1.0" encoding="utf-8"?> <!doctype plist public "-//apple//dtd plist 1.0//en" "http://www.apple.com/dtds/propertylist-1.0.dtd"> <plist version="1.0"> <dict>     <key>label</key>     <string>local.me.nytxwd</string>     <key>ondemand</key>     <true/>     <key>programarguments</key>     <array>         <string>/bin/sh</string>         <string>/users/me/documents/misc/nytimes_xword/nytimes_puzzle_dl4.sh</string>     </array>     <key>standarderrorpath</key>     <string>/users/me/documents/misc/nytimes_xword/err</string>     <key>standardoutpath</key>     <string>/users/me/documents/misc/nytimes_xword/tmp</string>     <key>workingdirectory</key>     <string>/users/me/documents/misc/nytimes_xword/</string> </dict> </plist> 

-=-=-=-=-=-=end .plist file-=-=-=-=-=-=

here observations:

-script runs fine command line (with necessary mods) -script fails when run launchd (exit 1) per-user job (~/library/launchagents/). system log reveals file grep looks , contains 2 essential bits of data never written target directory ("grep token login.html - no such file").

i tried run startcalendarinterval job @ 0900 every day when failed, switched on ondemand using launchctl load/unload debugging. somethings tried:

specifying paths in plist (error, out, working directory) - no help

changing ownership , permissions on target directory. - no help

changing paths in script absolute (no ~, no $home, no ., no relative). - no https://myaccount.nytimes.com/auth/login \ -o /users/me/documents/misc/nytimes_xword/login.html &>/dev/null>

i've read man pages launchd, launchctl, launchd.plist without epiphanies.

i've googled every combination of launchd/bash/wget , learned somethings still no success.

i ran example provided http://www.mactech.com/articles/mactech/vol.21/21.06/launchd/index.html , worked!! however, didn't illustrate how write , read text file user directory.

mac os x 10.5.8 if matters.

i know quite bit longer typical question posed here wanted avoid obvious "did try this" questions.

i'm assuming guru going reveal mojo of launchd simple , obvious solution. well, @ least i'll move on launchd step 2.

thanks, jlh

i have same issue. solved using curl instead.

wget http://opendata-download-metobs.smhi.se/api/version/1.0/parameter/2/station/178970/period/corrected-archive/data.csv

curl -c - -o data.csv http://opendata-download-metobs.smhi.se/api/version/1.0/parameter/2/station/178970/period/corrected-archive/data.csv

the first line worked me in terminal had use second in scripted version. 1 thing curl allowed me specify file name, wget had refused do.