Hi,

Two different custom collectors were developed (CC1 + CC2). Each
Collector had 1 DB Connector to 1 particular table/view (TBL1 and TBL2).
Each Collector was tested during development with 1 Event Source.
Testing went fine with an offset using the RowNumber (Oracle DB).
Switching to a lower offset, lead to getting correct rows from DB. The
development seemed to be correct. There was a moment in time (during
development) to decide not to use 2 DB-Queries in one collector (but
create 2 collectors with each 1 DB-Query). The Custom Collector reads
data straightforward with query and %s. Table column data is parsed and
mapped to event, tax and sev. No fancy stuff.

(In short: CC1 queries TBL1, CC2 queries TBL2).

However, implementing the collectors in production, lead to very strange
behavior in using offset and querying the databases. The Collectors were
gathering data from 2 databases (DB1 + DB2) spread over 2 servers
(Machines SRV1 and SRV2). De 2 DB are technically the same (same tables
etc.) but functionally the data in the tables are not the same.

Next setup in production:
CC1_______CN1_______ES1 (to SRV1 + DB1 + TBL1)
_____________________ES2 (to SRV2 + DB2 + TBL1)

CC2_______CN2_______ES1 (to SRV1 + DB1 + TLB2)
_____________________ES2 (to SRV2 + DB2 + TLB2)
This lead to an offset in both Connectors to use the lowest Offset in
both Event Source (ES1 + ES2). This lead to one connector stopping
getting data from one event source and the other connector looping over
and over again from the second event source. The two Collectors (CC1 +
CC2) did not influence each other (luckily). Only within each Collector,
the offset was influenced.

The following set up was tested also:
CC1_______CN1_______ES1 (to SRV1 + DB1 + TBL1)
__________CN2_______ES2 (to SRV2 + DB2 + TBL1)

CC2_______CN3_______ES1 (to SRV1 + DB1 + TLB2)
__________CN4_______ES2 (to SRV2 + DB2 + TLB2)
This second setup also led to an offset in both Connectors to use the
lowest Offset, in both Event Sources (ES1 + ES2). And again querying to
one connector stopped getting data from one event source and the other
one looping over and over again from the second event source. The two
Collectors (CC1 + CC2) did again not influence each other (luckily).
Only within each Collector, the offset was again influenced.

The only setup which did not mess up the offset was the following:
CC1_______CN1_______ES1 (to SRV1 + DB1 + TBL1)
CC1_copy__CN2_______ES2 (to SRV2 + DB2 + TBL1)

CC2_______CN3_______ES1 (to SRV1 + DB1 + TLB2)
CC2_copy__CN4_______ES2 (to SRV2 + DB2 + TLB2)

So we had to setup 4 collectors in ESM (in which 2 collectors were
copies). Connecting these copies with connectors to the event sources
lead to the correct use of the offset.

Tech data: Sentinel 7, Eclipse SDK 3.7.1, NetIQ Sentinel
SDK: 2011.1.1.v.20120419

What is happening here? Is this a mistake in building the collector? Did
I have to take something in consideration (technically while building
the custom collector)?
Please advice. The engineers, setting up the collectors in production,
do want to use the setup as was originally thought of (maintenance
reasons). And I cannot blame them. But for now: only setup 3 is working
as designed.


--
ggh_pennings
------------------------------------------------------------------------
ggh_pennings's Profile: https://forums.netiq.com/member.php?userid=5475
View this thread: https://forums.netiq.com/showthread.php?t=48868