i searching multiple patterns in file. file of 90gb in size, searching on particular field(from position 6-17 in each line). trying lines contain of particular list of numbers. current syntax using is:
grep '^.\{6\}0000000012345\|^.\{6\}0000000012543' somelargefile.txt > outputfile.txt for small number of patterns works. large number of patterns "argument list long" error.
one alternative have tried search each patters separately (using loop on patterns), require multiple passes on large data file(57102722 lines) not efficient.
from understand "argument list long" error, related bash cmds in general , not specific grep. there setting can used around error? or alternatively, ideas how using awk or sed or tool?
thank you!
you can avoid problem putting patterns in file, , using -f command line option grep.
the convenient put each alternative in separate line of file:
patterns.txt
^.\{6\}0000000012345 ^.\{6\}0000000012543 invocation
grep -f patterns.txt somelargefile.txt > outputfile.txt
Comments
Post a Comment