i found useful bit of perl here writes filename of text file first line of file. running terminal in os x yosemite:
perl -i -pe 'begin{undef $/;} s/^/\nfilename:$argv\n/' `find . -name '*.txt'` with modification thought had solved specific problem files i'm picking utf-16le , i've since discovered command writing in utf-8 , making real mess of output (text visibly correct not recognised in calculations in excel, filemaker etc).
after several attempts need getting script write filename in utf-16le start of file. (note: have workaround of batch convert files utf-8, run i'd prefer have workflow in 1 step).
reinierpost correct - more removing original unicode byte order mark (bom). worked in end was:
perl -i -pe 'begin{undef $/;} s/\xff\xfe/filename:$argv\n/' `find . -name '*.txt'` where utf-16le bom \xff\xfe replaced new string. reference other boms : - iso-10646-1 > \xfe\xff - utf-16be > \xfe\xff - utf-8 > \xef\xbb\xbf
i able write new text utf-16le with
perl -i -pe 'begin{binmode stdin,":encoding(utf8)";binmode stdout,":encoding(utf16)"; undef $/;} s/\xff\xfe/\xff\xfe\nfilename:$argv\n/' `find . -name '*.txt'` however believe source data mixed bag of utf8 , utf16 last version creates mixed set of characters between new header , data. reinierpost steering me in right direction. remain interested if others can improve this.
Comments
Post a Comment