bash - Glob through a long list of files with curl and writing each to a unique, new file with a matching filename -
i downloading several large sets of files ftp site using curl , writing them new files same name. want use command this:
curl ftp/site/filename > filename here example files ftp site:
file:compara.6_primates_epo.chr10_1.emf.gz 30909 kb 8/3/10 12:00:00 file:compara.6_primates_epo.chr10_2.emf.gz 13110 kb 8/3/10 12:00:00 file:compara.6_primates_epo.chr10_3.emf.gz 24948 kb 8/3/10 12:00:00 file:compara.6_primates_epo.chr10_4.emf.gz 45155 kb 8/3/10 12:00:00 file:compara.6_primates_epo.chr10_5.emf.gz 17236 kb 8/3/10 12:00:00 file:compara.6_primates_epo.chr11_1.emf.gz 37593 kb 8/3/10 12:00:00 ... 500 files later .... file:compara.6_primates_epo.other_9.emf.gz 8980 kb 8/3/10 12:00:00 there lot of these files, , going through many long lists of them. there some numerical patterns use
for x in {1..n}; curl ftp/site/file${x} > file${x}
but there unpredictable patterns, example secondary numeric index varies different files. {1..5} here:
compara.6_primates_epo.chr10_{1..5}
and next file might have
compara.6_primates_epo.chr11_{1..7}
and there oddballs last one:
compara.6_primates_epo.other_{1..?}
i know can't use simple globbing in:
curl ftp/site/file* > file* -- gives me ambiguous redirect. wondering if knew of clever solution glob filename, save string , write output filename, without getting too fancy... know go , write python script more systematically, seems there might quick bash solution failing think of...
the answer question simple after all:
curl -o ftp/path/to/site/*glob_pattern*
the -o option save files same name given in directory :-)
Comments
Post a Comment