Windows – Batch – How to cat multiple files together without header records


I'm attempting to merge multiple files together with the following command:

copy test1.txt+test2.txt test3.txt

This works just fine, however, the files I'm attempting to merge contain header records at the beginning of each one. Is there a command I can use to exclude the first line of a txt file, and only concatenate the rest?

EDIT: End goal is to combine six .txt files containing hundreds of thousands of records into a single file.

Best Answer

The following works well from the command line for most circumstanses

(for %F in (test1.txt test2.txt) do @more +1 "%%F") >test3.txt

Double up the percents if you use the command within a batch script.

The above has the following limitations

  • Each source file must have fewer than 64k lines, else it will hang.
  • Any tab characters will be converted into a string of spaces
  • I think there is at least one other restriction, but my memory fails me (perhaps null bytes converted to new line??)

The following batch script has no limitation other than each line must be less than 8k in length. But it is probably too slow for large files (batch is a lousy tool for text processing):

@echo off
setlocal disableDelayedExpansion
>test3.txt (
  for %%F in (test1.txt test2.txt) do for /f "skip=1 delims=" %%A in (
    'findstr /n "^" "%%F"'
  ) do (
    set "ln=%%A"
    setlocal enableDelayedExpansion

You could write a custom JScript or VB script that could do this efficiently.

My JREPL.BAT hybrid JScript/batch utility can handle this well. It is overkill, but it will efficiently do the job, even with very large files.

JREPL.BAT is a general purpose regular expression text processor, with many options. It is pure script that runs natively on any Windows machine from XP onward.

The following will work on the command line.

>test3.txt (for %F in (test1.txt test2.txt) do @JREPL "^.*" "ln>1?$0:false" /jmatch /f "%F")

If used within a batch script, then you must use CALL JREPL, and double the percents:

@echo  off
>test3.txt (for %%F in (test1.txt test2.txt) do call JREPL "^.*" "ln>1?$0:false" /jmatch /f "%%F")
Related Question