When we stop file connector and wait for some hours if some new
sequential files come to ES then File connector resumes/starts reading
from new file, it does not continue reading from the latest offset/file
after we start it. So that means we loose some log files, between latest
offset and new file's starting.

Event sources are on Windows 2003/2008/2012 and on all ESís result is
same it always switch to next file, it did not continue from latest
recorded offset.

Also if new file did not come to ES folder, file connector correctly
resumes from latest offset on current file, but if new one comes it
takes wrong decision.

For example, we stopped file today(November 1), at 4 pm. It was
collecting logs from an IIS server. We didn't start it again until
tomorrow(November 2) 8 am. So, it was not collecting logs for 16 hours.
Meantime, IIS server's log file is rotated. There was a file for
November 1, and our connector was reading it before we stopped it. At 12
a.m., the IIS server rotated the log file, and it started to write its
logs for a new file for November 2, when our connector were still
After these events, if we start the file connector at November 2, it
cannot resume from saved offset in the file for November 1, skips that
file, and it starts to read the file for November 2. So, it cannot read
logs from November 1, between 4 p.m (the moment when the connector
stopped) and 12 a.m (the moment when the IIS log file rotated from
November 1 to November 2).

Did someone see this problem before?

adilakbas's Profile: https://forums.netiq.com/member.php?userid=6009
View this thread: https://forums.netiq.com/showthread.php?t=49111