Home

Results 1 to 7 of 7

Thread: Apache HTTP

Hybrid View

  1. #1
    Join Date
    Dec 2007
    Location
    Melbourne, Victoria, Australia
    Posts
    1,304

    Apache HTTP

    I used to have this Connector running fine....then I decided it would be a good idea to update some plugins *shoot me now*. I've tried reverting to only "released" versions (Apache & Syslog), but still my events are mangled!

    Raw:
    Code:
    {"i_Second":"11","s_Date":"Apr 09 08:10:11","i_milliseconds":"1554761411000","i_TrustDeviceTime":"","i_DayOfMonth":"9","s_raw_message2":"<133>Apr  9 08:10:11 xxxxxx APACHE_HTTPD: www.isag.melbourne x.x.x.x - - [09\/Apr\/2019:08:10:04 +1000] \"GET \/media\/images\/favicon_16x16.ico HTTP\/1.1\" 200 99678 \"-\" \"Mozilla\/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko\/20100101 Firefox\/66.0\"","i_syslog_facility":"16","s_RV24":"B4F289F0-7F5F-1036-8B4A-000C294C00E8","s_RV25":"6349D9E9-3C75-1037-B6BD-000C294C00E8","s_RV22":"B4F289F0-7F5F-1036-8B40-000C294C00E8","s_RV23":"B4F289F0-7F5F-1036-8B48-000C294C00E8","s_RV21":"C76D2820-C395-1029-BB86-001321B5C0B3","CONNECTION_MODE":"map","s_SyslogRelayIp":"x:x:x:x:x:x:x:x","i_Hour":"8","sf":"","i_syslog_priority":"133","CONNECTION_METHOD":"SYSLOG","s_Version":"2019.1r1-201902270522-SNAPSHOT","s_Body":"APACHE_HTTPD: www.isag.melbourne x.x.x.x - - [09\/Apr\/2019:08:10:04 +1000] \"GET \/media\/images\/favicon_16x16.ico HTTP\/1.1\" 200 99678 \"-\" \"Mozilla\/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko\/20100101 Firefox\/66.0\"","i_Minute":"10","s_AppId":"APACHE_HTTPD","i_Year":"2019","s_MessageOriginatorHost":"xxxxxx","s_chainId":"1554704691183","s_sha256Hash":"f180ec1d8212525bc2a99db5866fb940f3997f9a21a2e128b0ab0906c95a8f60","i_Month":"3","i_syslog_severity":"5","s_chainSequence":"1558","s_MessageOriginatorPort":"54686","i_RXBufferLength":"244","i_Type":"2","EventSourceManagerID":"C76D2820-C395-1029-BB86-001321B5C0B3","CollectorID":"B4F289F0-7F5F-1036-8B40-000C294C00E8","EventSourceGroupID":"B4F289F0-7F5F-1036-8B48-000C294C00E8","EventSourceID":"B4F289F0-7F5F-1036-8B4A-000C294C00E8","EventRecordID":"6349D9E9-3C75-1037-B6BD-000C294C00E8","ChainID":"1554704691183","ChainSequence":"1558","EventDate":"04\/09\/2019 07:58:19.144 +1000","TenantID":"101100"}
    Event:
    Code:
    TargetHostClass(rv81),ObserverHostDepartment(obsdep),TargetHostFunction(rv82),SourceHostID(rv77),RetentionPolicyName(rv192),TargetHostDepartment(rv98),SourceHostGeoData(srcgeo),Severity(sev),TargetHostName(dhn),ObserverIP(obsip),SearchTargetID(rv172),TargetHostCriticality(rv84),RetentionPolicyID(rv171),TargetIP(dip),TargetServiceName(dp),TenantID(tid),EventTime(dt),ObserverTZMonth(estzmonth),CollectorNodeName(port),SourceHostName(shn),VendorOutcomeCode(voc),ObserverHostClass(obsclass),SourceIP(sip),ReporterIP(repip),ReporterHostID(repassetid),EventID(id),TargetHostLongitude(dlong),ObserverHostGeoData(obsgeo),Vulnerability(vul),SentinelProcessTime(spt),TargetHostGeoData(destgeo),ObserverTZDayInYear(estzdiy),EventName(evt),SentinelServiceID(src),ObserverTZDayInWeek(estzdiw),ObserverTZDayInMonth(estzdim),SourceHostLongitude(srclong),ProductName(pn),SentinelProcessingComponent(rt2),ObserverHostFunction(obsfunc),ObserverHostName(sn),ObserverType(st),TenantHierarchyID(rv1),CollectorPluginName(agent),IdTApprovedAccountAdmins(cv81),IDManagedSystems(cv82),TargetHostLatitude(dlat),ObserverHostLongitude(obslong),NetworkZone(cv97),SentinelID(rv121),CollectorPluginID(rv122),ObserverHostCriticality(obscrit),Message(msg),ObserverTZHour(estzhour),SourceHostLatitude(srclat),TargetHostCountry(rv30),ObserverCategory(rv32),MinRetentionDate(rv164),ObserverHostLatitude(obslat),ObserverTZ(estz),TenantName(rv39),ConnectorID(rv23),ObserverTZMinute(estzmin),CollectorID(rv22),RawDataRecordId(rv25),EventSourceID(rv24),CollectorManagerID(rv21),SourceHostCountry(rv29),Tags(rv145),ObserverHostCountry(obscountry)
    Physical,ISAG,Main Server,0,System Events,ISAG,"-37.8330862,144.9455179",4,xxxxxx,x.x.x.x,B4F289F0-7F5F-1036-96D5-000C294C00E8,Critical,6E1CCA35-4BD4-102D-91CD-000C2907C76D,192.168.245.3,httpd,101100,Tue Apr 09 07:58:19 AEST 2019,3,Apache HTTPD,www.isag.melbourne,GET /media/images/favicon_16x16.ico HTTP/1.1,Physical,x.x.x.x,x:x:x:x:x:x:x:x,0,BD49D9E9-3C75-1037-B677-000C294C00E8,144.9455179,"-37.8330862,144.9455179",0,Tue Apr 09 07:58:19 AEST 2019,"-37.8330862,144.9455179",99,+1000],B4F289F0-7F5F-1036-8B40-000C294C00E8,3,9,144.9455179,Apache HTTP Server,Apache HTTP Server,Main Server,xxxxxx,A,0,Apache HTTP Server,0,0,-37.8330862,144.9455179,LAN,B4F289F0-7F5F-1036-9632-000C294C00E8,A5E13B30-5A4A-102C-9069-005056C00008,Critical,+1000] by www.isag.melbourne,7,-37.8330862,AU,WEB,Mon Jul 08 10:00:00 AEST 2019,-37.8330862,Australia/Melbourne,ISAG,B4F289F0-7F5F-1036-8B48-000C294C00E8,58,B4F289F0-7F5F-1036-8B40-000C294C00E8,6349D9E9-3C75-1037-B6BD-000C294C00E8,B4F289F0-7F5F-1036-8B4A-000C294C00E8,C76D2820-C395-1029-BB86-001321B5C0B3,AU,Sentinel,AU
    I know its hard to spot, but it appears that the date parsing is causing the problems....it doesn't seem to cope with the " +1000" GMT identifier, so all the fields get offset and its just plain rubbish.

    I grabbed the SDK and tried looking, but am completely lost. I think its in the release.js (???) line 282:

    Code:
    var evtDate = DateTime.parseExact(this.fields[3].substr(1), "dd/MMM/yyyy:HH:mm:ss", this.fields[4].substr(0, 5));
    But unsure, and only reference to parseExact is Microsoft's C# one, but this is JS....so...????
    Visit my Website for links to Cool Solution articles.

  2. #2
    Join Date
    Dec 2007
    Location
    Melbourne, Victoria, Australia
    Posts
    1,304

    Re: Apache HTTP

    I take that back, I don't think it's date...

    Code:
    LogFormat "%v %a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""
    That fails....but I have something on the network that seems to hit this format:

    Code:
    LogFormat "%a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""
    So, I think its issues in parsing the Virtual Host in the log file....but I really need that field to identify which host as I have several hosts sending through.
    Visit my Website for links to Cool Solution articles.

  3. #3
    klasen is offline Micro Focus Employee - Ultra Contributor
    Join Date
    Nov 2007
    Location
    Germany
    Posts
    355

    Re: Apache HTTP

    On 2019-04-09 00:34, ScorpionSting wrote:
    > So, I think its issues in parsing the Virtual Host in the log
    > file....but I really need that field to identify which host as I have
    > several hosts sending through.


    The collector was designed to parse the vhost from the log file name.
    Did you setup your directory layout according to
    https://www.netiq.com/support/sentin...ration_section
    ?

    NOTE: The actual file names of the log files is not critical (adjust the
    file pattern appropriately), but that the virtual domain must be in the
    path two levels up from the log files to be captured so that the
    Collector can properly determine the virtual domain name

    --
    Norbert

  4. #4
    Join Date
    Dec 2007
    Location
    Melbourne, Victoria, Australia
    Posts
    1,304

    Re: Apache HTTP

    Quote Originally Posted by klasen View Post
    On 2019-04-09 00:34, ScorpionSting wrote:
    > So, I think its issues in parsing the Virtual Host in the log
    > file....but I really need that field to identify which host as I have
    > several hosts sending through.


    The collector was designed to parse the vhost from the log file name.
    Did you setup your directory layout according to
    https://www.netiq.com/support/sentin...ration_section
    ?

    NOTE: The actual file names of the log files is not critical (adjust the
    file pattern appropriately), but that the virtual domain must be in the
    path two levels up from the log files to be captured so that the
    Collector can properly determine the virtual domain name

    --
    Norbert
    Using rsyslog...custom log file for each vhost, so access/error isn't actually used by main logging (just all the other c**p that tries to http)...
    Last edited by ScorpionSting; 09-Apr-2019 at 11:02 PM.
    Visit my Website for links to Cool Solution articles.

  5. #5
    Join Date
    Dec 2007
    Location
    Melbourne, Victoria, Australia
    Posts
    1,304

    Re: Apache HTTP

    Quote Originally Posted by klasen View Post
    On 2019-04-09 00:34, ScorpionSting wrote:
    > So, I think its issues in parsing the Virtual Host in the log
    > file....but I really need that field to identify which host as I have
    > several hosts sending through.


    The collector was designed to parse the vhost from the log file name.
    Did you setup your directory layout according to
    https://www.netiq.com/support/sentin...ration_section
    ?

    NOTE: The actual file names of the log files is not critical (adjust the
    file pattern appropriately), but that the virtual domain must be in the
    path two levels up from the log files to be captured so that the
    Collector can properly determine the virtual domain name

    --
    Norbert
    Not sure why this worked previously and only started to misbehave after I started playing with plugin updates....I haven't changed apache vhost logging config for some time...

    Its just a complete pain having to configure directories just for Sentinel then having to retrofit that change into awstats, logrotate, etc...

    Would be nice if there was some way to configure the collector to map the file path to the log syntax using the apache % parameters... (i.e. /path/to/apache.log = "%v %a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"")...then have the release.js use that for mapping.
    Visit my Website for links to Cool Solution articles.

  6. #6
    klasen is offline Micro Focus Employee - Ultra Contributor
    Join Date
    Nov 2007
    Location
    Germany
    Posts
    355

    Re: Apache HTTP

    Hi,

    in comparison to the combined LogFormat

    Code:
    LogFormat    "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\"
    \"%{User-agent}i\"" combined
    the collector was written to handle, your has an extra field (%v) at the
    beginning:

    Code:
    LogFormat "%v %a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""
    You could try moving it to the end and then assign this.fields[10] to
    this.domain in customParse()

    Otherwise you need to customize Record.prototype.parseAccess()

    --
    Norbert

  7. #7
    Join Date
    Dec 2007
    Location
    Melbourne, Victoria, Australia
    Posts
    1,304

    Re: Apache HTTP

    Quote Originally Posted by klasen View Post
    Hi,

    in comparison to the combined LogFormat

    Code:
    LogFormat    "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\"
    \"%{User-agent}i\"" combined
    the collector was written to handle, your has an extra field (%v) at the
    beginning:

    Code:
    LogFormat "%v %a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""
    You could try moving it to the end and then assign this.fields[10] to
    this.domain in customParse()

    Otherwise you need to customize Record.prototype.parseAccess()

    --
    Norbert
    Thanks Norbert
    Visit my Website for links to Cool Solution articles.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •